Understanding of Automation Testing and Automation Framework What is Automation Testing?Automation Testing is a process where software or an Automation Tool is used to execute test scripts in a controlled and desired manner to compare the expected result with the actual result. In general terms Automation Testing involves automating a manual process which is already in place in the testing process.Automation Fundamental ConceptsPrepared By: Priya Trivedi
Automation Fundamental Concepts Automation life cycle is a subset of the entire test life cycle Automation planning can be initiated in parallel to the test planning phase Factors to be considered in automation planning,Stability of AUT (Application under test)No of regression cycles to be performedCompatibility of App platform with testing toolsCost benefit analysis (ROI)Availability of skilled resources When Automation is applicable?Regression Testing Cycles are long and iterative.If the application is planned to have multiple releases / buildsIf its a long running application where in small enhancements / Bug Fixes keeps happeningTest Repeatability is required
The Benefits/Purpose of Automation testingSpeed up Test Execution: In less span of time you can run more no of test scripts which not only reduces the time but also speeds up the execution task.Achieved Improved Test Coverage: Your coverage is more than what you could do manually.Ensures Consistency, Accuracy and Efficiency: The tests run are consistent, accurate and efficient.Reliability and Transparency: The tests can perform the exact same steps every time without concern about human error.Repeatability and Reusability: Once you automate your test cases i.e. you convert the test cases into test scripts they can be run any number of times since they are stored and they can be reused at a later point of time when needed.Return on Investment (ROI)Quality
What Should Be AutomatedGood candidatesTests executed for each buildBusiness critical testsTests that are difficult/tedious to perform manually
Bad candidatesTests without predictable resultsTest that require variable input/responses from the testerTests that perform operations in multiple environments
Tool NameCompany NameLatest VersionHP QTP HP10.5IBM Rational Functional Tester IBM Rational188.8.131.52SeleniumOpen Source Tool1.0.1SilkTestMicro Focus2009TestCompleteAutomated QA7.52TestPartnerMicro Focus6.3WATIROpen Source Tool1.6.5
Introduction to Test Automation Framework What an Automation Framework is ? How many types of Automation Framework are there ? How to select & Deploy an Automation Framework ? What are the Constituents of Automation Framework ?Areas covered Include:
What a Test Automation Framework is ?It is a structured combination of :
Various Assumptions for Testing Testing Concepts Testing Practices
Aim is to provide support to Automated Software Testing
Utility of Test Automation Framework Provides an Outline of overall Test Structure Ensures Consistency of Testing Minimizes the Amount of Code for Development - thereby Less Maintenance Maximizes Reusability Reduces Exposure of Non-Technical Testers to Code Enables Test Automation using Data
How Many Types of Automation Frameworks are there ? Data Driven Automation Framework
Keyword Driven Automation Framework
Modular Automation Framework
Hybrid Automation Framework
Generally there are 4 Types :
Data Driven Automation Framework
Advantages: Repeated use of Test Scripts with Different Inputs and Response Data coming out of Predefined Dataset Helps in Reducing Coding for Large Test Cases Ease of Testing of Time-Consuming & Complex Test Cases
Data Driven Automation FrameworkData-driven testing is a framework where test input and output values are read from data files (datapools, ODBC sources, csv files, Excel files, DAO objects, ADO objects, and such) and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. The test scripts contain all the coded information regarding reading of the data files, navigation through the application & test status logs etc. This has a similarity with table-driven testing in a way that our test case is encapsulated in the data file as against in the test script. The test script just remains a "driver," or a means of delivery for the data. Here in data-driven testing, data files contain just the test data only.
Continue..Merits of data-driven testingScripts may be developed while application development is still in progress Utilizing a modular design, and using files or records to both input and verify data, reduces redundancy and duplication of effort in creating automated test scriptsIf functionality changes, only the specific "Business Function" script needs to be updatedData input/output and expected results are stored as easily maintainable text records. Functions return "TRUE" or "FALSE" values to the calling script, rather than aborting, allowing for more effective error handling, and increasing the robustness of the test scripts. This, along with a well-designed "recovery" routine, enables "unattended" execution of test scripts.
Demerits of data-driven testingRequires proficiency in the Scripting language used by the tool (technical personnel)Multiple data-files are required for each Test Case. There may be any number of data-inputs and verifications required, depending on how many different screens are accessed. This usually requires data-files to be kept in separate directories by Test CaseTester must not only maintain the Detail Test Plan with specific data, but must also re-enter this data in the various required data-filesIf a simple "text editor" such as Notepad is used to create and maintain the data-files, careful attention must be paid to the format required by the scripts/functions that process the files, or script-processing errors will occur due to data-file format and/or content being incorrect
Keyword Driven Automation FrameworkAttributes: As the Name suggests, it enables Keyword Driven Testing or Table Driven Testing Data & Keyword Tables being Independent of the Automation Tool Enables Documentation of the Functionality of the Application under Test (AUT) in A Tabular Format
Keyword-driven testing is a Software testing technique that separates much of the programming work from the actual test steps so that the test steps can be developed earlier and can often be maintained with only minor updates, even when the application or testing needs change significantly.
The Keyword Driven framework consists of the basic components given below1. Control File2. Test Case File3. Startup Script4. Driver Script5. Utility Script
Keyword Driven Automation Framework
Control File Consists details of all the Test scenarios to be automated User will be able to select a specific scenario to execute based on turning on or off a flag in the Control File Control File is in the form of an excel worksheet and contains columns for Scenario ID, Execute (Y/N), Object Repository Path, Test Case File PathTest Case File Contains the detailed steps to be carried out for the execution of a test case It is also in the form of an excel sheet and contains columns for Keyword, Object Name, ParameterStartup Script The Startup script is utilized for the initialization and reads the control files It then calls the driver script to execute all the scenarios marked for execution in the control fileDriver Script It Reads the Test Case files. Checks the keywords and calls the appropriate utility script functions based on specific keyword Error Handling is taken care of in the driver script.Utility Scripts Perform generic tasks that can be used across applications. It should not be application dependentKeyword Driven Automation Framework
Keyword Driven Automation FrameworkAdvantage of Framework.The main advantage of this framework is the low cost for maintenance. If there is change to any test case then only the Test Case File needs to be updated and the Driver Script and Startup script will remain the same. No need to update the scripts in case of changes to the applicationDisadvantages of keyword driven testing:Development of "customized" (Application-Specific) Functions and Utilities requires proficiency in the tools Scripting language. (Note that this is also true for any method)If application requires more than a few "customized" Utilities, this will require the tester to learn a number of "Key Words" and special formats. This can be time-consuming, and may have an initial impact on Test Plan Development. Once the testers get used to this, however, the time required to produce a test case is greatly improved.
Modular Automation FrameworkAttributes: As the Name suggests, it enables Keyword Driven Testing or Table Driven Testing Data & Keyword Tables being Independent of the Automation Tool Enables Documentation of the Functionality of the Application under Test (AUT) in A Tabular Format
Test Script Modularity Framework:
Enables creation of Small, Independent Scripts representing Modules & Functions of the Application under Test (AUT). The use of this framework will yield a higher degree of modularization and add to the overall maintainability of the test scripts
Test Library Architecture Framework:
Enables creation of Library Files representing Modules & Functions of the Application under Test (AUT). This framework requires the creation of library files (SQABasic libraries, APIs, DLLs, and such) that represent modules, sections, and functions of the application-