- Automation Testing- Rational Functional Tester
Understanding Automation TestingWeb Automation ChallengesRoad map to Success & best PracticeIBM Rational Functional TesterAutomation Framework
What?Test automation is the use of software to control the execution of tests. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.Why?Reduce the impact on the bottom line in Regression Testing.Increase the flexibility of time and resource.Fast and Accuracy.When?Frequency build and regression testing on demand.Mission critical business & long term maintenanceTo avoid Human errors.To check real time performance of the application.GUI FrozenConfidentialAutomation Testing What? -Why? -When?
Test Automation is simple, that every tester can do it
This myth is promoted by the tool sales people. They are trying to promote the following test automation process:
Record the scriptEnhance the script by adding functions and data drivingRun the scriptsReport results
Under the influence of this myth the QA manager can proudly report: All our testers are developing test automation.
ConfidentialMyth # 1
Test automation is a software development task
Automation should be designed, developed and testedYou need to have some kind of a programming background to implement test automation. Test Automation is not as complex as C++ or C# or Java or VB development.Test automation standards should be developedAutomated test components are assets that should be treated like application source codeConfidentialMyth #1 - The Reality
Commercial test tools are expensive
Under the influence of this myth some companies, especially the small ones:
Try to develop their own test automation toolsUse scripting languages like Perl and RubyUse shareware test toolsDo not consider test automation at allConfidentialMyth #2
Commercial tools are cheap
Per seat license for the most expensive automation tool is $8K.This tool will be used for 5 years.Maintenance/Support fees are 20% of tool cost or $1,800 per year.The cost of this tool is $8K/5+$1,800 = $3,100 per year.The automation developer cost with overhead is $100K per year.The cost of this tool is just 3% of the person who uses it, but productivity gain can be very significant.ConfidentialMyth #2 - The Reality
Customer support. Many of the open source tools come and go with little to no support .Most commercial tools are constantly being updated as technologies change.Most commercial tools usually have more functionality (IBM RFT & QTP can test various GUI applications: Web, .Net, Java, VB, C/C++, Power Builder, etc. vs. WATIR Web only)Commercial tools usually have a large community of users, which translates into better availability of qualified resourcesCommercial tools require less advanced programmingMore test automation frameworks are available for commercial toolsCommercial tools are integrated with Test Management tools which makes reporting and execution much simpler.ConfidentialCommercial Benefits
Over 300 Test Tools are available in market (http://www.softwareqatest.com)
Load/Performance tools 54Web Functional/Regression 60Java Test tools 48Other Web tools 76
Which tool is right for you?ConfidentialCommercial Tools
ConfidentialDice search results across US (30 days, Jan 2010)
ConfidentialDemand Per Tool
Excerpt from IDC Report:-
Worldwide Automated Software Quality 2007-2011 Forecast and 2006 Vendor Shares
Courtesy :- http://www.idc.com/
International Data Corporation (IDC)ConfidentialAutomation is Future !
ROI value IS NOT the value of Automation vs. Cost of executing these tests manually
Automation ROI value IS the benefit of this type of testing, and it can be:Reducing Time to Market.Increased Test Efficiency (Productivity)Increased Test Effectiveness.ConfidentialClassic ROI Calculation
Can get a greater market shareMakes people available to work on other projectsHigher margins, if no competitive products are currently availableReduce time to market
ConfidentialROI helps to ..
More testing gets done faster, increasing the odds of finding defects.
Defects found early have better chances of being fixed.
Manual Testers can concentrate on clever ways to finding defects, instead of typing test inputs and verify output.
About 7% of bug fixes create new bugs, sometimes in already tested parts of the system. With automation you can rerun tests for those modules. This almost never happens when testing is done manually.ConfidentialROI - Productivity and effectiveness
Each project requires different types of automation - there is often no easy formula available to calculate ROI
Performing ROI calculation can help to determine upfront what type of automation, what level of skills, what tools will be required.
- The Next slide may help you to understand simple ROI calculation.ConfidentialROI Summary
Is Automation is expensive ? Yes of course ! Its expensive and time consuming effort at initial stage.
Functionality testingGUI.Business validation.Data Base.
Performance TestingStress TestLoad TestVolume TestConfidentialAutomation Capabilities
Are you thinking of 100% Automation ?
Impossible ..Stop Dreaming.Certainly you have areas like performance testing, regression testing, load/stress testing where you can have chance of reaching near to 100% automation. Areas like User interface, documentation, installation, compatibility and recovery where testing must be done manually.
Will your automation suite be having long lifetime?
Every automation script suite should have enough life time that its building cost should be definitely less than that of manual execution cost.Approximately your automation suite should be used or run at least 20 to 30 times for separate builds (General assumption. depends on specific application complexity) to have good ROI.ConfidentialGeneral qualms
Is Tool Selection more Important ?Be sure to have skilled staff before deciding to automate your testing work.Make sure that tool is a best fit to your requirementsFind out the limitations of the tool that is best match with your requirements
Why web application is different in Test Automation Perspective ?
Many web based commercial system like B-C, B-B & C-CCatering to large number of end user.People with varying background & technical skills may use the application.Network thread (Security , Connection speed (Broad band , dial-up etc.. ))Numerous application usage.Browser issue (IE ,Mozilla fire-fox etc.., )ADA (American with disability act) (The Application may need to be tested for compliance and usability).Money & Time Zone Factor (Standards).Hence a lot of effort needs for Test Planning and Test Design.ConfidentialWeb Automation
Rule 1: Get familiar with Business.Rule 2: Get familiar with Tool.Rule 3: Have at least one real programmer in your team.Rule 4: Set some standards.* Naming standards* Coding standards* Environment standards* Procedure for error and defect tracking system* Documentation standards.*Source safe.Rule 5: Figure out what you are testing and keep simple.Rule 6: Start with small mile stone.Rule 7: Modularize and build re usability into your scripts.Rule 8: Document Everything.Confidential8 Rules for Successful Automation
Processor : Pentium- 4 , 1.4 + GHz (HT)RAM : Min 1 GB Recommended 2GB OS : Win 2x, XP, Hard Disk Space : 1GBJava : JRE 1.4 and above.ConfidentialIBM Rational Functional TesterIBM Rational Functional Tester for Java
Introduction to Functional Tester
Working with a Functional Test Project
Setting up the Test Environment
Rational Functional Test is an object-oriented automated testing tool that tests Java, HTML, VB.NET, and Windows applications, and lets you record reliable and robust scripts that can be played back to validate new builds of a test application.
Functional Test runs on Windows 2000, Windows XP, and Linux platforms ConfidentialIntroduction to Functional Tester
Functional Test is available in two integrated development environments and two scripting languages. Functional Test, Java Scripting uses the Java language and the IBM Rational Software Development Platform. Functional Test, VB.NET Scripting uses the VB.NET language and the Microsoft Visual Studio .NET development environment.
ConfidentialIntroduction to Functional Tester (Cont..)
With the object-oriented recording technology of Functional Tester, you can generate scripts by running and using the application-under-test. Functional Tester scripts are implemented as Java programs.
Functional Tester is integrated with IBM Rational Test Manager, which enables you to record and play back a Functional Tester script from Test Manager and make use of Test Manager test logs. Alternatively, Functional Tester can be used as a standalone product, using HTML or text logs to review test