- Automation Testing
- Rational Functional Tester
∗ Understanding Automation Testing∗ Web Automation Challenges∗ Road map to Success & best Practice∗ IBM Rational Functional Tester∗ Automation Framework
Confidential
Agenda
What?∗ Test automation is the use of software to control the execution of tests. Commonly,
test automation involves automating a manual process already in place that uses a formalized testing process.
Why?∗ Reduce the impact on the bottom line in Regression Testing.∗ Increase the flexibility of time and resource.∗ Fast and Accuracy.When?∗ Frequency build and regression testing on demand.∗ Mission critical business & long term maintenance∗ To avoid Human errors.∗ To check real time performance of the application.∗ GUI Frozen
Confidential
Automation Testing –What? -Why? -When?
“ Test Automation is simple, that every tester can do it”
∗ This myth is promoted by the tool sales people. They are trying to promote the following test automation process:
∗ Record the script∗ Enhance the script by adding functions and data driving∗ Run the scripts∗ Report results
∗ Under the influence of this myth the QA manager can proudly report: All our testers are developing test automation.
Confidential
Myth # 1
“Test automation is a software development task”
∗ Automation should be designed, developed and tested∗ You need to have some kind of a programming
background to implement test automation. Test Automation is not as complex as C++ or C# or Java or VB development.
∗ Test automation standards should be developed∗ Automated test components are assets that should be
treated like application source code
Confidential
Myth #1 - The Reality
“Commercial test tools are expensive”
∗ Under the influence of this myth some companies, especially the small ones:
∗ Try to develop their own test automation tools∗ Use scripting languages like Perl and Ruby∗ Use shareware test tools∗ Do not consider test automation at all
Confidential
Myth #2
“Commercial tools are cheap”
∗ Per seat license for the most expensive automation tool is $8K.∗ This tool will be used for 5 years.∗ Maintenance/Support fees are 20% of tool cost or $1,800 per
year.∗ The cost of this tool is $8K/5+$1,800 = $3,100 per year.∗ The automation developer cost with overhead is $100K per year.∗ The cost of this tool is just 3% of the person who uses it, but
productivity gain can be very significant.
Confidential
Myth #2 - The Reality
∗ Customer support. Many of the open source tools come and go with little to no support .
∗ Most commercial tools are constantly being updated as technologies change.
∗ Most commercial tools usually have more functionality (IBM RFT & QTP can test various GUI applications: Web, .Net, Java, VB, C/C++, Power Builder, etc. vs. WATIR – Web only)
∗ Commercial tools usually have a large community of users, which translates into better availability of qualified resources
∗ Commercial tools require less advanced programming∗ More test automation frameworks are available for commercial tools∗ Commercial tools are integrated with Test Management tools which makes
reporting and execution much simpler.
Confidential
Commercial Benefits
Over 300 Test Tools are available in market (http://www.softwareqatest.com)
∗ Load/Performance tools – 54∗ Web Functional/Regression – 60∗ Java Test tools – 48∗ Other Web tools – 76
Which tool is right for you?
Confidential
Commercial Tools
Confidential
Dice search results across US (30 days, Jan 2010)
Confidential
Demand Per Tool
Excerpt from IDC Report:-
∗ “Worldwide Automated Software Quality 2007-2011 Forecast and 2006 Vendor Shares…”
Courtesy :- http://www.idc.com/
International Data Corporation (IDC)
Confidential
Automation is Future !
∗ ROI value IS NOT the value of Automation vs. Cost of executing these tests manually
Automation ROI value IS the benefit of this type of testing, and it can be:∗ Reducing Time to Market.∗ Increased Test Efficiency (Productivity)∗ Increased Test Effectiveness.
Confidential
Classic ROI Calculation
∗ Can get a greater market share∗ Makes people available to work on other projects∗ Higher margins, if no competitive products are
currently available∗ Reduce time to market
Confidential
ROI helps to …..
∗ More testing gets done faster, increasing the odds of finding defects.
∗ Defects found early have better chances of being fixed.
∗ Manual Testers can concentrate on clever ways to finding defects, instead of typing test inputs and verify output.
∗ About 7% of bug fixes create new bugs, sometimes in already tested parts of the system. With automation you can rerun tests for those modules. This almost never happens when testing is done manually.
Confidential
ROI - Productivity and effectiveness
∗ Each project requires different types of automation - there is often no easy formula available to calculate ROI
∗ Performing ROI calculation can help to determine upfront what type of automation, what level of skills, what tools will be required.
∗ - The Next slide may help you to understand simple ROI calculation.
Confidential
ROI Summary
∗ Is Automation is expensive ? Yes of course ! Its expensive and time consuming effort at initial stage.
Confidential
ROI
∗ Functionality testing∗ GUI.∗ Business validation.∗ Data Base.
∗ Performance Testing∗ Stress Test∗ Load Test∗ Volume TestConfidential
Automation Capabilities
∗ Are you thinking of 100% Automation ?
∗ Impossible ..Stop Dreaming…….∗ Certainly you have areas like performance testing,
regression testing, load/stress testing where you can have chance of reaching near to 100% automation.
∗ Areas like User interface, documentation, installation, compatibility and recovery where testing must be done manually.
∗ Will your automation suite be having long lifetime?
∗ Every automation script suite should have enough life time that its building cost should be definitely less than that of manual execution cost.
∗ Approximately your automation suite should be used or run at least 20 to 30 times for separate builds (General assumption. depends on specific application complexity) to have good ROI.
Confidential
General qualms
∗ Is Tool Selection more Important ?∗ Be sure to have skilled staff before deciding to automate
your testing work.∗ Make sure that tool is a best fit to your requirements∗ Find out the limitations of the tool that is best match with
your requirements
Confidential
General qualms
∗ Why web application is different in Test Automation Perspective ?
∗ Many web based commercial system like B-C, B-B & C-C∗ Catering to large number of end user.∗ People with varying background & technical skills may use
the application.∗ Network thread (Security , Connection speed (Broad band ,
dial-up etc.. ))∗ Numerous application usage.∗ Browser issue (IE ,Mozilla fire-fox etc.., )∗ ADA (American with disability act) (The Application may
need to be tested for compliance and usability).∗ Money & Time Zone Factor (Standards).∗ Hence a lot of effort needs for Test Planning and Test
Design.
Confidential
Web Automation
∗ Rule 1: Get familiar with Business.∗ Rule 2: Get familiar with Tool.∗ Rule 3: Have at least one real programmer in your
team.∗ Rule 4: Set some standards.
∗ * Naming standards∗ * Coding standards∗ * Environment standards∗ * Procedure for error and defect tracking system∗ * Documentation standards.∗ *Source safe.
∗ Rule 5: Figure out what you are testing and keep simple.
∗ Rule 6: Start with small mile stone.∗ Rule 7: Modularize and build re usability into your
scripts.∗ Rule 8: Document Everything.
Confidential
“8” Rules for Successful Automation
System Requirements
∗ Processor : Pentium- 4 , 1.4 + GHz (HT)∗ RAM : Min 1 GB Recommended 2GB ∗ OS : Win 2x, XP, ∗ Hard Disk Space : 1GB∗ Java : JRE 1.4 and above.
Confidential
IBM Rational Functional TesterIBM Rational Functional Tester for Java
∗ Introduction to Functional Tester
∗ Working with a Functional Test Project
∗ Setting up the Test Environment
Getting Started…..
∗ Rational Functional Test is an object-oriented automated testing tool that tests Java, HTML, VB.NET, and Windows applications, and lets you record reliable and robust scripts that can be played back to validate new builds of a test application.
∗ Functional Test runs on Windows 2000, Windows XP, and Linux platforms
Confidential
Introduction to Functional Tester
∗ Functional Test is available in two integrated development environments and two scripting languages. Functional Test, Java Scripting uses the Java language and the IBM Rational Software Development Platform. Functional Test, VB.NET Scripting uses the VB.NET language and the Microsoft Visual Studio .NET development environment.
Confidential
Introduction to Functional Tester (Cont..)
∗ With the object-oriented recording technology of Functional Tester, you can generate scripts by running and using the application-under-test. Functional Tester scripts are implemented as Java programs.
∗ Functional Tester is integrated with IBM Rational Test Manager, which enables you to record and play back a Functional Tester script from Test Manager and make use of Test Manager test logs. Alternatively, Functional Tester can be used as a standalone product, using HTML or text logs to review test results.
Confidential
Functional Tester Features
Confidential
Script Recording and Playback Process
Confidential
Starting Functional Tester
Click Start > Programs > IBM Rational Software Development Platform >
Functional Tester V6.1 > Java Scripting
∗ The workbench is the desktop development environment that lets you work with multiple tools seamlessly. The workbench contains one or more perspectives. A perspective provides a set of functionality for working with specific tasks or types of resources. You switch between different tasks and resources by selecting your perspective.
∗ Perspectives define the initial workspace configuration and layout.
∗ It Provides a focused working environment.
Confidential
Perspectives
Confidential
Perspectives (Cont..)
Switch from one perspective to another to carry out a different task
Perspectives can be opened
In the same (existing) window In a new window In the same window, replacing the current perspective
Confidential
Perspectives (Cont..)
∗ The perspectives can be opened in the same window or in a new workbench window. For example, if you want to see Functional Test Debug at the same time as the Java editor, you can open the Functional Test Debug perspective in a separate window.
∗ If you always want to open a perspective in its own window, you can set it up as a user preference. To do this, click Window > Preferences > Workbench > Perspectives and then, under Open a new perspective, select In a new window.
Confidential
Perspectives (Cont..)
∗ To Open a Perspective:∗ From the Functional Test perspective, click Window >
Open Perspective or click the Open a perspective button.
∗ Click one of the listed perspectives or click Other to display a list of perspectives .(The Functional Test Debug and Java perspectives are listed by default, but this menu changes depending on which perspectives are open.)
∗ In the Select Perspective dialog box, click the perspective you want to open and then click OK.
Confidential
Perspective (Cont..)
The Functional Test Projects view lists test assets for each project including:
∗ Folders∗ Scripts∗ Shared test object maps∗ Log folders∗ Logs∗ Java filesConfidential
Functional Test Perspective
Confidential
Functional Test Perspective (Cont..)
∗ Views are an important component of your workspace. Views display different kinds of information within a perspective.
To open a view, click Window > Show View and select a view from the list.Commonly used views include the following:∗ Tasks view displays errors, warnings, or other information automatically generated by the script.∗ Search view displays results from a search.∗ Console view displays output from the script or application (such as print statements or exceptions).∗ Bookmarks view displays a list of markers that point to a specific place in
the workbench.∗ Breakpoints view lists all the breakpoints you have set in your script. If
you press the F1 key while you are in this view, you can access Help information about breakpoints.Confidential
Views
∗ The Functional Test Project stores Functional Tester application test assets including:
∗ Scripts
∗ Object maps
∗ Verification point baseline files
∗ Script templates
∗ Datapools
Confidential
Working with Functional Test Project
Confidential
Connecting to a Functional Test Project
1)Click File > Connect to a Functional Test Project.
2)Specify the project location path and Project name.
3)Click Finish.
∗ It is important to have a consistent context for the each testing cycle.
∗ Context includes:∗ Hardware testing configuration∗ Database∗ Network considerations∗ State of the application-under-test (AUT)∗ Sequence of actions
Confidential
Setting up the Test Environment
∗ Before using Functional Tester to test HTML applications the WEB BROWSERS should be enabled
∗ Before using Functional Tester to test Java applications the JREs Should be enabled.
Confidential
Enabling the Environment for Testing
Confidential
Enabling Web BrowsersIn the Menu, Click Configure>Enable Environments for Testing
select Web Browsers tab and click Search and after selecting the browser click Finish.
43
Configuring Applications for Testing1)In the menu Click Configure>Configure Application for Testing
2)To add a new application Click Add button and after providing Name and Path of the application click Finish.
3)To remove an application select the application and click Remove
∗ Create a script and start recording
∗ Perform user actions in the AUT
∗ End recording
Recording a Script
Confidential
Create a script and start recording
1)Click Record a Functional Test script on the tool bar
2)Select the Project folder
3)Specify the name of the new script
4)Click Finish
∗ The monitor displays message for every action performed during recording session such as:
∗ Starting and stopping the recording
∗ Starting an application or browser
∗ Clicking and all other actions upon an application
∗ Inserting verification points and other itemsConfidential
Recording Monitor
Confidential
Recording a Script1)In the Recording Tool bar Click
the Start Application icon
2)Select the Application and click OK
∗ To perform user actions in the application under test:
∗ Record∗ The actions (mouse clicks, keystrokes) as moving
through the AUT
∗ Verification points to capture and save information about specific objects
Confidential
Performing User Actions in the AUT
Confidential
End RecordingAfter Performing User actions on the AUT click Stop
Recording icon on the Recording tool bar
The Verification point and Action wizard can be used to
1)Select an object to test in the application2)Select an object to perform on the object
Confidential
Recording Verification Points
To open the Verification Point and Action Wizard, click the Insert Verification Point
or Action Command button on the Recording toolbar.
∗ When a verification point is inserted into a script, it capture information about the state of the AUT so that it can verify that state in subsequent test cycles. Functional Test stores the information that captured first as the baseline of expected behavior.
∗ When you play back the script against a new build of the AUT, Functional Test recaptures the information about the AUT state and compares it to the baseline. If the information captured at playback time has a value within the expected tolerance, the verification point passes. If there are differences, the verification point fails.
Confidential
Recording Verification Points (Cont..)
Three methods to select an object
1)Drag Hand Selection (Object Finder)
2)Test Object Browser
3)Time Delayed Selection
Confidential
Select an Object to Test
Confidential
Selecting an Object-Drag Hand Selection
1) Select Drag Hand Selection in the Selection Method drop-
down menu of the
Select an Object page in the Verification Point and Action
Wizard.
2) Grab the Object Finder tool icon with your mouse and drag it over the object
that you want to select. The object is outlined with a red border, and the object
name is displayed.
3)When mouse button is released, the object is selected and its recognition
properties are listed in the Object Recognition Properties grid at the bottom of
the page.
Confidential
Selecting an object-Test Object Browser
To select an object by using the Test Object Browser tool:
1. Select Text Object Browser in the Selection Method drop-down menu of the
Select an Object page in the Verification Point and Action Wizard.
2. Browse the object tree until you find the object that you want to select.
3. Click the object you want to test. The object is selected, and its recognition
properties are listed in the object recognition properties grid.
Confidential
Selecting an Object-Time Delay Method
To select an object by using the Delay Method:
1)Select Time Delayed Selection in the Selection Method drop-down menu of the
Select an Object page in the Verification Point and Action Wizard.
2) Set the number of seconds. The default is 10 seconds.
3) Click the Object Finder icon.
4) Move your mouse to hover over your application until you get to the object you
want to select. Anything you do during the delay period is not recorded.
5) When the timer runs out, the object under the cursor is selected and its
recognition properties are listed in the grid.
Confidential
Select an Action to perform against an Object
After the object is selected, click the Next button to select an action to perform
against the object.
There are four types of actions that can be performed against the object
selected to
test. Two of the actions are verification points (Data or Properties) and two are
scripted actions against the object.
∗ Script support functions are used to insert code into the Functional Test script while recording.
∗ Call Script inserts a call Script command into your Functional Test script.
∗ Comment inserts a comment into the script.∗ Log Entry inserts a log message into the script.
During playback, Functional Test inserts this information into the log.
∗ Sleep inserts a sleep command into the script to delay a single action.
∗ Timer inserts a timer into the script and stops the timer. Timers remain running until you stop them explicitly or exit Functional Test.
Confidential
Script Support Functions
Confidential
Script Support Functions (Cont..)To add script support features to your script:
If you are recording a script, click the Insert Script Support Commands button
on the Recording toolbar.
If you are editing a script:
a. Position the pointer in the script where you want to insert the feature.
b. Click the Insert Recording into Active Functional Test script button on the
Test Perspective toolbar.
c. Click the Insert Script Support Commands button on the Recording
toolbar.
∗ Insert Recording to an existing script is done to:
∗ Correct a recording mistake
∗ Resume recording after an interruption
∗ Test a newly added feature
Confidential
Adding to an Existing Script
Confidential
Insert Recording to existing scripts
From toolbar:
To insert recording into an existing script:
1. To correct a recording mistake, use the mouse to select the mistake.
Then press the Delete key to delete the mistake.
2. Position the cursor in the part of the script where the new recording is to
go.
3. Insert recording into the current script in one of the following ways:
Click Script > Insert Recording
Click the Insert recording into Active Functional Test script button
4. When the Functional Test Recording Monitor appears, recording can be
started at
the current cursor position.
Rational Functional Tester
Our Mission :To offer our customers a true assessment of software readiness.
Playback & Results Analysis
To play back a script and view playback results, you should be able to:
∗ Run a Functional Test script∗ Identify the different types of logs∗ View verification point results∗ Describe the script debugging process∗ Identify the main Functional Test debugging tools∗ Set Functional Test playback and logging options
Confidential
Objectives
Confidential
Script Recording and Playback Process
∗ Two phases:Test development
∗ Verify that the script works as intended.∗ Validate baseline of expected behavior.
Regression testing∗ Compare latest build to baseline established in test
development phase.
Confidential
About Playing Back a Script
1. Select the script.2. Click the Run Functional Test Script button.
Confidential
Initiating Playback
Confidential
The Playback Monitor
Confidential
Playback Log Types
Confidential
Playback Log Types: HTML Log
Confidential
Playback Log Types: Text Log
Confidential
Comparison of Playback Log Types
∗ The Verification Point Comparator allows you to compare the baseline data with the actual results you captured when you played back the script against the AUT.
∗ Use the Verification Point Comparator to:Compare verification point data after you play back a script with a verification point Update the baseline file
∗ To open the Verification Point Comparator:• In an HTML log, click the “View Results” hyperlink to open the VP comparatorfor a VP that failed. (If the VP passed, the VP Editor is opened, displaying theexpected results.)• For a text log, right-click the log and then click Failed Verification Points.
Confidential
Verification Point Comparator
Confidential
Verification Point Comparator Display
∗ A failure reveals a difference between the expected and actual results
∗ Investigate all failure∗ Use the Verification Point Comparator to determine differences∗ Update baseline data if necessary
∗ Product enhancements∗ Design changes
∗ Report defects if necessary
Playing back a script can reveal several different types of failures. These failures are normal and don’t necessarily mean that there’s anything wrong with your script. If failures occur during regression testing when you play back against a different build of the application-under-test, it usually means that your script is doing its job correctly If failures occur during script development when you play back against the same build of the AUT, it usually means that your script needs some simple debugging.
Confidential
When a Verification Point Fails
1. Verify the playback environment.2. Identify the symptom.3. Locate the cause of the error.4. Resolve the script error.Debugging: Verify Environment and Identify Symptom∗ Is the playback environment the same as the recording
environment?∗ Verify the test configuration.∗ Watch the script play back.∗ Check the log.∗ Could it be a timing issue?
Confidential
Solution: Debug the Script
∗ Play back the script.∗ Observe playback.∗ Examine the log.
∗ How much of the playback completes?∗ What types of errors are there?
∗ Examine the script.∗ Step through script in the Debug perspective∗ Examine script in the Java editor
Confidential
Debugging: Locate the Cause of the Error
∗ Compilation failures∗ Error messages during playback∗ Verification point failures∗ Script command failures∗ Synchronization failures
Confidential
Debugging: Types of Failures
Functional Test debugging tools include:∗ Playback log∗ Functional Test Debug perspective∗ Debug View∗ Verification Point Comparator∗ Breakpoints∗ Stepping∗ Stop() command
Confidential
Functional Tester Debugging Tools
Confidential
Debugging Tools: Debug Perspective
Confidential
Debugging Tools: Breakpoints
Confidential
Debugging Tools: Stepping
∗ Playback options∗ Playback delay options∗ Playback monitor∗ Time options multiplier∗ Logging optionsFunctional Test Playback Options∗ Maximum time to attempt to find Test Object∗ Pause between attempts to find Test Object∗ Skip Verification Points∗ Timeout used in waitForExistence() method∗ Retry time used in waitForExistence() loop
Confidential
Solution: Set Functional Test Options
∗ To change Functional Test playback settings:1. Select Window > Preferences to display the Preferences dialog box.2. In the Preferences dialog, expand Functional Test in the left pane.3. Select the Playback preferences page.4. Change the settings. Clear the Use Default check box in order to edit the setting.5. Click Apply to save the new setting and continue making changes or click OK to save the new setting and close the Preferences dialog box.
Confidential
Setting Functional Test Playback Options
Confidential
Setting Functional Test Playback Options
∗ Delay before mouse up∗ Delay before mouse move∗ Delay before mouse down∗ Delay before key up∗ Delay when hover∗ Delay after top-level window activates∗ Delay before key down∗ Delay before performing Test Object action
Confidential
Functional Test Playback Delay Options
Confidential
Setting Functional Test Playback Delays
∗ To change Functional Test playback delays:1. Select Window > Preferences to display the Preferences dialog box.2. In the Preferences dialog, expand IBM Rational Functional Tester in the left pane.3. Expand Playback in the left pane.4. Select the Delays preferences page.5. Change the settings. Clear the Use Default check box to edit the setting.6. Click Apply to save the new setting and continue making changes or click OK to save the new setting and close the Preferences dialog box.
Confidential
What to do
Confidential
Setting Functional Test Playback Monitor Option
∗ Use this Preference dialog to specify whether to display the playback monitor during playback.
∗ To change the Functional Test playback monitor setting:1. Select Window > Preferences to display the Preferences dialog box.2. In the Preferences dialog, expand IBM Rational Functional Tester in the left pane.3. Expand Playback in the left pane.4. Select the Monitor preferences page.5. Change the setting.6. Click OK to save the new setting and close the Preferences dialog box.
Confidential
What to do
∗ Multiply all time options by this number Applies to:∗ Playback∗ Delays∗ Recorder
This setting is useful for debugging and for resolving synchronization problems. For example, if you slow playback, you can watch more closely to determine the cause of a playback error. In addition, if the script is playing back too quickly for the AUT, you can slow playback to give the AUT time to “catch up” to the script. This can help to eliminate errors caused by the absence (or late appearance) of an object upon which the script needs to act.
For example, if you are playing back a script on a relatively slow computer and set
the time options multiplier to two, the script will take twice as long to play back.
Confidential
Functional Test Time Options Multiplier
Confidential
Setting the Functional Test Time Options Multiplier
∗ To change Functional Test preferences for the current Functional Test user:1. Select Window > Preferences to display the Preferences dialog box.2. In the Preferences dialog, expand Functional Test in the left pane.3. Clear the Use Default check box so you can edit the value.
4. Enter any real number by which to multiply all Functional Test preferences that take an amount of time as an argument.5. Click OK to save the new setting and close the Preferences dialog box.
Confidential
What to do
∗ Use default log information (do not prompt for log information)
∗ Display log after script playback∗ Prompt before overwriting an existing log∗ Log type
∗ None (no log created)∗ Text∗ HTML (default)
Confidential
Functional Test Logging Options
Confidential
Setting Functional Test Logging Options
∗ To set Functional Test logging options:1. Select Window > Preferences to display the Preferences dialog box.2. In the Preferences dialog, expand Functional Test in the left pane.3. Select the Logging preferences page.4. Change the options.5. Click Apply to save the new setting and continue changing options or click OK to save the new setting and close the Preferences dialog box.
Confidential
What to do
Rational Functional Tester
Our Mission :To offer our customers a true assessment of software readiness.
Extending Scripts
To extend scripts, you should be able to: Identify the main components of a Functional Test
script.
Edit a script generated by Functional Tester.
Modify a script to use a message box.
Modify a script to override preferences.
Modify a script to handle an unexpected active window.
Confidential
Objectives
Confidential
Class Hierarchy of a Functional Tester Script• Rational Test Script provides base-
level functionality and serves as the root of the inheritance hierarchy for all Functional Tester scripts.• Optional, user-written helper super classes provide override support for base level methods, particularly the event handler methods.• The script helper class provides script-specific methods for accessing Test Objects and Verification Points in the AUT.• The Functional Tester script contains the recorded or scripted commands that make up a specific functional test.
Confidential
Sample Functional Tester Script
A Functional Tester script includes four main categories of statements: Methods that this script inherited from
RationalTestScript (such as startApp or logTestResult) Method calls on test objects (such as Click or Drag) Statements that perform verification points Any additional Java code you want that is not generated
by Functional Tester
Starting Applications Under Functional Tester The first statement in a Functional Tester script that
has not been manually modified is frequently the one that contains startApp. References a symbolic name for the application under
test Example: public void testMain (Object[] args) { startApp
(“Classics JavaA"); .... }
Confidential
Contents of a Functional Tester Script
The object reference in a user-action statement can take two arguments: Anchor (the context in which the user-action is to be performed) State (enabled, disabled, showing, not_showing,loaded, loading, uninitialized,
ready)
Example 1:ProcessTestObject pid =StartApp(“ClassicsJavaA");PlaceOrder(pid, DEFAULT) .click();Example 2:Close(ANY, MAY_EXIT). click();
Verification Points Verification point statements appear in the script with the name that the tester
assigns to the verification point when it is recorded. The name of the verification point is always followed by VP(). performTest();
Example:Order_Total_AmountVP(). performTest();Confidential
Replaying User Actions (cont.)
Script Editing Features
Confidential
Editing a script
Confidential
Integrated Development Environment (IDE) Features
Use the Java editor to edit a script. The name of the script you are currently editing appears in a tab
at the top of the frame. An asterisk (*) on the left side of the tab indicates that there are unsaved changes.
You can open several files in the Java editor and move among them by clicking the appropriate tab.
If there is a problem with the code, a problem marker is displayed near the affected line.
Right-click the Java editor to display menu options for working with scripts.
The Problems view displays errors, warnings, and other information automatically generated by the compiler. To open the Problems view, click the Problems tab in the Test
perspective or click Window > Show View > Basic > Problems. When you double-click a Problems item, Functional Tester
positions the cursor at the corresponding line of code in the script.
By default, this view lists all problems for all files in the project. Click the Filter button to apply a filter.
Confidential
Integrated Development Environment (IDE) Features
Confidential
Creating a message box1. Import the JOptionPane class:import javax.swing.JOptionPane;
2. Specify and display the message:JOptionPane.showMessageDialog(null,”Application starting.”,”Information”,JOptionPane.INFORMATION_MESSAGE);
Customizing Your Message Box:
JOptionPane.showMessageDialog(null,”Application starting.”,”Information”,JOptionPane.INFORMATION_MESSAGE);
INFORMATION_MESSAGE
WARNING_MESSAGE
ERROR_MESSAGE
Confidential
Overriding Preference Settings
• The package com.rational.test.ft.script provides the classes that manage test scripts.• IOptionName contains definitions of the customization options you can use.
Confidential
Overriding Preference Settings (cont.)
The example sets the value of TIME_MULTIPLIER to 10.
Challenge: Dialog Box Sometimes Pops UpDuring testing, the browser occasionally pops up a
security settings dialog.The dialog does not always come up, so I do not want
the script to always click a button.How can my script handle this event?
Confidential
Handling an Unexpected Active Window
A common problem in GUI testing is the appearance of an unexpected activewindow. If you click the OK button in the window during recording, the click isrecorded. If the window does not appear during playback, the OK button is notfound and the script fails. If you remove the button click from the script, the scriptwill fail when the window does appear during playback. How should you handle this situation?
Implement the onTestObjectMethodException event Add a method to use this event Call a helper super script to handle the event for any
Functional Test script that may handle the condition Handle the event whenever it occurs
For details and sample code, refer to the Functional Test Help, “Handling Unexpected Active Windows.”
The onTestObjectMethodException event is expected to handle most of the conditions that occur with an unexpected active window. If the dialog is not modal or the window cannot be found, you will have to use another event (or events).
Confidential
Solution: Handle Unexpected Active Window
Use a method with this eventUse the WindowActivateFailedException exceptionUse the Iwindow interfaceAdd code as necessary
Confidential
Handling an Unexpected Active Window
Confidential
What to doTo modify a script so that it handles an unexpected active window:
1. Add a method to override the onTestObjectMethodException event.
2. Add an if statement to see if the WindowActivateFailedException exceptionoccurred. Inside this loop, call the IWindow interface to get information on theactive window that is causing the problem.
3. Add an if statement under the call to the IWindow interface to see if the activewindow is present. If it is present, send an Enter key command to it. After theactive window receives the Enter key command, execute the command on thedesired object again.
4. Add code to call the helper class if the window cannot be found or it cannot beactivated.
Rational Functional Tester
Our Mission :To offer our customers a true assessment of software readiness.
Test Object Maps
∗ As Applications undergo changes throughout the course of development, the Automated scripts can ‘break’.
∗ Testers must constantly update the scripts to handle minor changes in the application.
∗ Instead of updating the script code, the Test Object map can be updated. All Scripts that reference this object map can ‘see’ the update.
Introduction
Confidential
Test Object Map
Test Object Map is a hierarchical collection of descriptions of test objects.
Contains properties (name-value pairs) for each test object Recognition properties enable Functional Tester to find the test object during playback. Administrative properties describe the test object.
While performing recording action in an AUT it creates a Test script and the corresponding Test Object Map. The Test Object Map can be seen in the Script Explorer Perspective
A Script is always associated with a single test object map
A test object map can be:Private
Associated with only one scriptAutomatically created while recording an AUT
SharedCan be associated with more than one scriptAutomatically updated when recording a script
Confidential
Scripts and Test Object Maps
Confidential
Private Test Object Map
Private test object map is used to isolate test object differences within a single script.
Associated with only one script
Displayed by the script explorer
Deleted when the script is deleted
Confidential
Shared Test Object Map
Shared test object map is used to share the test objects along multiple scripts.
A Private test object map cannot be converted to a Shared test object map. Instead a Private test object map can be merged to a new or existing Shared test object map.
Confidential
Displaying a Test Object MapTo open a shared test object map from the Project Explorer:Double-click the test object map Right-click the test object map and select Open Test Object MapTo open a test object map (private or shared) from the Script Explorer: Double-click the test object map Right-click the test object map and select Open Click the Open Test Object Map button on the toolbar
Confidential
Creating a Shared Test Object MapTo create a test object map that can
be shared:
1. On the menu bar, click File > New > Test Object Map.
2. In the Create a Test Object Map dialog, specify the following and then click Next.
Location of the test object map Name of the map Whether you want to add the map
to Clear Case version control Whether you want this test object
map to be the default choice for new Functional Tester scripts
Confidential
Creating a Shared Test Object (Cont..)
(optional)
3. In the Copy Test Objects to New Test Object Map dialog, select the way you want to populate the new test object map:To create an empty map with no objects, choose Don’t copy any TestObjects.To create a map using an existing test object map as a template, click SelectTest Object Maps and scripts to copy Test Objects from and then selectthe maps and scripts you want to base the new map on.If you have selected scripts and want to associate those scripts with the newmap, click Connect selected Functional Tester scripts with new Test Object Map.4. Click Finish.
Confidential
Associating a script with a Shared Test Object MapTo associate a new Functional Tester
script with a shared test object map: Click Record a Functional Test script. Complete the Record a Functional Test script dialog and click Next. In the Select Script Assets dialog box, click the Test Object Map Browse button and select the map to be associated with the script. In the Select Test Object Map dialog, select the shared test map you wish toassociate with the new script and then click OK. Click Finish in the Select Script Assets dialog.
Confidential
Adding an object to a Test Object Map1.In the test object map menu, click
Applications > Run and select the applicationthat contains the object you want to add to the test object map. This opens theapplication. (If necessary, add the application.)2. Click Test Object > Insert Object(s).... Functional Tester displays the Insert aGUI Object into the Object Map dialog box.(continued)
3. In the Insert a GUI Object into the Object Map dialog box, click the Object Finder icon and drag it over the object you want to add to the test object map.
Confidential
Adding an object to a Test Object Map (Cont..)4. After selecting the object to add, its properties will be displayed. Click Next tocontinue.5. On the Select Object Options page, specify what to include:Just the selected object All siblings of the selected object All available objects in the window6. Click Back to add more objects or click Finish.7. In the Test Object Map, click File > Save to save the updated test object map.
Confidential
Adding Test Objects to a scriptTo add a test object to a script:1. Click Test Object > Associated Scripts.2. Select the scripts to which you want to add the test object and then click OK.3. Click Test Object > Add to Script [name of script1]... Functional Tester adds the test object to the Script Explorer of the selected script.
4. In the script, position the cursor where you want to add the object.5. In the Script Explorer, right-click the object you want the script to reference. In the drop-down menu, select Insert at Cursor. Functional Tester displays a list of all possible methods for the object.6. Double-click the method you want to use. Functional Tester inserts the object and the method you selected into the script at the cursor location.
Confidential
Updating Properties of Test Objects To edit object properties:
1. In the Test Object Map, select the object.2. In the property sets pane, double-click the value you want to edit. Propertiesthat cannot be edited are labeled “(read only).“
To add descriptive text to an object’s Administrative property set:1. Right-click the object and select Description Property.2. In the Set Description Property dialog, enter the text you want to use for theobject‘s description property. If a description property already exists for this test object, you can edit it.3. Click OK.
Confidential
Deleting an Object from a Test Object MapTo remove an object from a test
object map:1. In the test object map, select the test object to be deleted.2. Do one of the following: Right-click and select Delete. On the test object map toolbar, click the Delete button.3. In the Delete Test Object dialog box, do one of the following: Click Next to list all of the scripts that reference the test object and that will be affected by the deletion. Click Finish to delete the object from the test object map. Click Cancel to close the dialog box without deleting the test object.
Rational Functional Tester
Our Mission :To offer our customers a true assessment of software readiness.
Managing Object Recognition
Recognize a Test Object
Properties of an object during recording
Exact Match of the properties of the object during Playback
Confidential
Object RecognitionWhen one property does not match Functional tester plays back the script without issuing a warning message
When two properties does not match the object is still found, but a ‘weak recognition’ warning is written to the log
If there are too many differences the object cannot be found
∗ Each object found during playback is assigned a recognition score.∗ Greater recognition score means less exact match.Examples:∗ Exact Match: Score=0
∗ One property with weight of 100 does not
match: Score=10,000∗ Two properties that do not match: Score=20,000
Confidential
Object Recognition Scores
The goal of this mechanism is to enable playback of scripts despite significant changes to objects in the application under test.
∗ Setting thresholds for recognition scores enables to control Functional Tester
object-matching sensitivity during play back.
∗ Functional Tester uses these recognition score thresholds when searching
for objects during script playback:
∗ Maximum acceptable recognition score determines the maximum score an
object must have in order for Functional Tester to recognize it as a candidate.
Any objects with higher recognition scores are not considered until the timeout
has expired.
∗ Warn if accepted score is greater or equal to causes a “weak recognition”warning to be written to the log if Functional Tester accepts a candidate
with ascore greater than or equal to the score specified in this field. A score of
10000means that one important property can be wrong.
Confidential
Thresholds for Object Recognition Scores
∗ Last chance recognition score specifies the maximum acceptable score an
object must have for Functional Tester to recognize it as a candidate if a suitable match is not found after Maximum time to attempt to find Test Object has elapsed.
∗ Ambiguous recognition score difference causes an Ambiguous Recognition Exception to be generated if the scores of top candidates differ by less than the amount specified in this field. If two objects are seen ast he same, the difference between their scores must be at least this number for Functional Tester to prefer one object over the other.
Confidential
Thresholds for Object Recognition Scores (cont..)
Confidential
Recognition Score Thresholds-Standard
Recognition level determines how tolerant Functional Tester is of differences between an object and the recognition properties for which Functional Tester is seeking a match. Move the slider to the right (Strict) to require a closer match or to the left (Tolerant) to permit a match even though there are differences.
Warning level determines the point at which Functional Tester reports a difference to the test log. Move the slider to the right if you want Functional Tester to report relatively small differences or to the left if you want Functional Tester to report only significant differences.
Confidential
Recognition Score Thresholds-AdvancedTo change Functional Tester playback recognition score thresholds:1. Select Window > Preferences to display the Preferences dialog box.2. In the Preferences dialog, expand Functional Test in the left pane.3. Expand Playback in the left pane.4. Select the Script Assure (tm) preferences page.5. Change the settings. Clear the Use Default check box in order to edit a setting.6. Click Apply to save the new setting and continue making changes, or click OK to save the new setting and close the Preferences dialog box.
Rational Functional Tester
Our Mission :To offer our customers a true assessment of software readiness.
Data-Driven Test
∗ Data-driven testing is useful when tests:
Can separate test steps from input data Must be repeated many times using different data
each time
∗ For example, create a data-driven test to:
Test the boundaries of a quantity field in an online retail application
Test variable length data in a text input field Test the order total functionality of a retail cash
register
Data-Driven Test
Confidential
Datapools A datapool: Is a collection of related data records
Supplies data values to the variables in a test script during script playback
Automatically provides a different set of test data each time you play back a test
In Functional tester:Datapools can be created while recording a scriptImport existing datapools
Edit the created datapools within Functional Tester
Export and edit datapools
Share a datapool with multiple test scripts
Confidential
Creating a Data-Driven Test
Confidential
Creating a Data-Driven Test (Cont..)
Confidential
Changing a Verification Point Reference to a Datapool
When recording a verification point, Functional tester captures object information and literal values from the AUT and stores it as a baseline for comparison during playback. However, a verification point baseline can be changed from a literal value to a datapool reference. Then, when the script plays back, the verification point references the datapool and uses the variable data as the base line for comparison.
Confidential
Importing DatapoolsFunctional Tester allows importing an external datapool and use it to drive a test script. The data can be imported from:
An external spreadsheet (.csv file)
Another Functional Test datapool
An existing IBM Rational Test Manager datapool
To import data from a Test Manager datapool, the Functional Tester project must be associated with the Rational project that contains the datastore
Confidential
Importing Datapools (Cont..)
Confidential
Exporting a Datapool
A Datapool from a Functional Tester project can be Exported to a .csv file
Exporting a datapool allows to: Add data to the datapool using an external application Use the datapool in a different Functional Tester project
Confidential
Running a Data-Driven Test
When playing back the script, set the datapool iteration count. Each time the script runs, it pulls a different record from the datapool. When a particular count or select Iterate Until Done is specified, it means that the script will run once for each record in the datapool.
Confidential
Data-Driven Testing ScenariosScenario 1 Create a datapool when recording a data-driven script within Functional Tester and then modify the datapool within Functional Tester.
Scenario 2 Import an externally created datapool into Functional Tester and then associate it with a a data-driven test script.
Scenario 3 Create a datapool when recording a data-driven script within Functional Tester. Export the datapool and edit it externally. Import the edited datapool to drive a test script.
Confidential
Creating a Data-Driven Test Script and Datapool
To record a data-driven script and create a datapool:
Begin recording a script
Insert data-driven commands into the script
Edit data-driven commands
Insert verification points and have them reference of theDatapool
When recording is finished, edit the datapool
During play back of the test script, set the number ofiterations
Confidential
Importing and Exporting DatapoolTo import an external datapool and associate it with
a test script:
Import the external datapool into Functional Tester
Check that the datapool imported correctly
Edit the datapool variable row to have meaningful variable names
Associate the datapool with an existing script or record a script and associate the datapool with it
Edit verification points to reference variable data in theDatapool
Find and replace literal values in the script with variables
Play back the script to test the datapool
In this round-trip scenario:
Record a data-driven test script with a datapool
Export the datapool (.csv file)
Edit the datapool in an external application
Import the edited datapool into the project
Associate the datapool with a new test script
Add data-driven commands to the new test script
Play back the script to test the datapool
Confidential
Exporting, Editing, and Importing a Datapool
∗ Automated database testing is faster and more reliable the traditional GUI-based test automation.
∗ Resource Competency.∗ Familiar with the basic SQL commands∗ Hands of experience / Exposure in Manual DB Testing∗ Have an interest in automating some of that testing
Confidential
Effective Data Base Testing
∗ The types of database activity (insert, delete, update) that will be invoked
∗ When these transactions will occur within the application-under-test ∗ The calculations or processing rules used by the application ∗ Any time-critical transactions, functions, conditions, and possible
causes of poor performance ∗ Data fields to ensure proper data types ∗ For proper spacing and length ∗ Data formats that may not be enforced by a data type ∗ For duplicate data ∗ Reference data to ensure that the data you are expecting is there ∗ Data migration Test Phase
Confidential
What Can be Test ?
Confidential
Data Base Jar connectivity
Confidential
∗ What is Automation Frame work ? ! ?∗ A test Automation Frame work is a set of assumptions, concepts, and
practices that provide support for automated software testing.∗ Data Driven approach is suitable for application that have limited
functionality but large number of variation in the terms of test data,∗ There is no hard and fast rule to use a specific automation frame work. It all
depends on your project needs , here are some info on the same.∗ Functional frame work is suitable for application that have variety of
functionality but limited variation in terms of test data.∗ Hybrid Test automation frame work is suitable for application that have
variety of functionality and larger number of variation in terms of test data.∗ Record enhance and playback methodology is suitable to convert small
medium size scripts in to equivalent automation scripts – one to one basis
Confidential
Frame work
∗ Modular∗ Data-Driven∗ Keyword–Driven∗ Model-Based
Confidential
Theoretical Frame work Names
∗ The Modular framework is the natural progression derived from Record-and-Playback
∗ •The modular framework seeks to minimize repetition of code by grouping similar actions into “modules”
∗ (e.g.: login)∗ •Test Data is in a script or in an internal table ∗ This is what tool sales people are promoting.
Confidential
Modular
∗ These frameworks are similar in that the data is separated from the test script
∗ • The script is just a "driver" or delivery mechanism for the data.
∗ •The difference:∗ - In keyword-driven testing, the navigation data and
test data are contained in the data source∗ - In data-driven testing, only test data is contained in
the data source.
Confidential
Keyword Driven / data Driven
∗ Model-based testing is software testing in which test cases are derived in whole or in part from a model that describes some (usually functional) aspects of the system.
∗ •Model-based testing for complex software systems is still an evolving field.
Confidential
Model Based
∗ Reduced the cost and time spent maintaining and updating tests
∗ •The modular structure of keyword-driven testing means that new tests can easily be created from pre-existing modules
∗ •The test team is capable of entirely automating tests, even without programming knowledge
∗ •Can be easily modified to use with different test tool∗ •Re usability across different projects
Confidential
Key word driven Adventages
Confidential
Confidential
Automation Process
Confidential
Sample schedule / Planning
Confidential
Thank you Vijay C N
Recommended
