V E R I F I C A T I O N
software quality optimization
SilkTest Reviewers Guide
This document is designed to assist you in your evaluation of SilkTest from Segue Software. SilkTest is the industrys leading functional and regression testing solution. The Reviewers Guide contains tips on what to look for in an automated testing solution and describes how SilkTest can help you begin or advance your automated testing efforts. If you are new to SilkTest or have minimal experience, you will want to start at the beginning, with Maximize Your Test Investment on page 3. You may be especially interested in the following sections: What to Look for in an Automated Testing Solution on page 3 and Developing Automation on page 9. If you have automation experience, you may want to pay particular attention to the following sections: Developing Automation on page 9, Moving Automation to the Next Level: Advanced Techniques on page 11 and Maintaining Automation: Coping with Change on page 12.
ABOUT THE SILKCENTRAL SOFTWARE QUALITY OPTIMIZATION (SQO ) PLATFORMTM TM
Segue offers SilkCentral - an integrated platform to define, measure, manage and improve software quality from development through deployment and into production. The SilkCentral platform provides a single point of control into all quality activities and up-to-the-minute visibility into application quality status, thus integrating all quality stakeholders throughout the software application lifecycle. SilkCentral serves as the foundation for Segues SQO strategy and incorporates all of the existing Silk family of products, as well as other third-party solutions, into one, cohesive environment. Segues highly respected Silk product line is fully integrated into SilkCentral, each test automation product serving as an engine to drive a specific quality function within the platform. For example, functional/regression testing and load testing are provided by Segues flagship products, SilkTest and SilkPerformer, respectively.
The SilkTest Reviewers Guide contains the following:
3 4 7 9 11 12 13 14 15
Maximize Your Test Investment Why Choose SilkTest How SilkTest Works Developing Automation Moving Automation to the Next Level Maintaining Automation: Coping with Change Running Tests and Analyzing Results Software Quality Optimization (SQO) throughout the Application Lifcycle Specific SilkTest Integrations
SilkTest is an easy-to-use test automation solution that enables users to be highly productive in their software test automation efforts.
SilkTest Reviewers Guide
M A X I M I Z E YO U R T E S T I N V E S T M E N TManually testing software is a time-consuming and often tedious process, one that cannot guarantee testing consistency across releases and platforms. Additionally, time constraints hinder the ability to manually test and retest applications before they are released. Inevitably the question remains: Did any critical bugs go undetected? of whether or not any errors occur during test execution. This means no one needs to spend time overseeing the test runs. Tests that are portable across platforms eliminate the need to spend time tweaking the tests to adjust to differences that the various configurations might introduce.
Why Automated Testing?A viable alternative or supplement to manual testing is automated testing. Automated tests offer several advantages, primarily productivity and repeatability. Automation provides a reduction in the time taken to perform testing and an increase in the predictability of test runs. With manual testing, human error can lead to falsely identifying defects. Another advantage of automating tests is that multiple tests can be generated quickly from a single, base test. Automation tools offer programmatic aids such as data-driven testing to generate multiple similar tests using different data, whereas using manual testers to create large numbers of repetitive tests is impractical. Automated testing leads to better use of both your QA staff and computer resources. Testers are freed up to put more effort where it counts - writing and accumulating new tests to expand test coverage of both new and existing application functionality. The vast majority of defects are found when developing new tests. While QA personnel are writing new tests, computers can be used to run unattended automated tests around the clock.
What to Look for in an Automated Testing SolutionThe key to an automated testing tool is how effective it is in helping you quickly determine application quality. This means that the automated testing tool should ensure that: Tests can run unattended regardless of how the application behaves during test execution Tests can run unmodified on any supported platform in need of testing Tests can run as soon as possible after the software is available Results are analyzed in a timely manner The consistent benefit in all of these factors is increasing productivity. True unattended testing does not require human intervention to keep the test suite running, regardless
Tests that are insulated from application changes, both at the interface and functional level, require little or no time to modify in order to run right away. This is an important point, given that the applications you need to test are continually evolving; if they were not you would no longer be testing them. Lastly, accurate, easily accessed and analyzed test result information is the final piece to timely quality assessment.
KEY BENEFITSLeverage existing skill sets to achieve high levels of automation productivity Maximize testing opportunities by running tests unattended 24x7 Reduce the risk of application errors by maximizing test coverage Meet demanding release schedules with shortened test cycles
SilkTest Reviewers Guide
WHY CHOOSE SILKTESTSilkTest has many benefits and features that will appeal to users of any skill level, but let us focus on the features that are most influential in helping you make timely, application quality assessments: Built-in, customizable Recovery System for unattended testing Flexible, robust, object-oriented 4Test scripting language, enabling you to test across multiple platforms, browsers and technologies with one set of scripts Agents for simultaneous distributed testing Reporting Direct database access and validation Workflow elements for test creation and customizationAOL NETSCAPE NAVIGATOR
For example, different browsers, such as Microsoft Internet Explorer, Netscape Navigator and AOL, have similarly named buttons that perform the same function, such as the button for refreshing the current page. With SilkTest, in most cases, you can use the same scripts and window declarations to interact with the application, regardless of which browser is being used to run the application. BrowserMICROSOFT INTERNET EXPLORER
Customizable Recovery SystemSilkTest has a built-in, highly customizable Recovery System that lets tests run unattended, regardless of the behavior of the Application Under Test (AUT). After the completion of each individual test in a series to be executed, the Recovery System restores the application to an easily defined initial or base state, including restarting the application in the event of an application crash.
The object-oriented aspects of 4Test, as well as other features such as window tagging (see Application GUI and SilkTest Objects on page 7 for more information), provide opportunities for abstraction of data and functions so that when the application changes, tests can be run with minimal modification. For example, the relocation of a link on a Web page would require no test modification. If the link was changed to a button, reclassifying the element from a link to a button, which is done in seconds with 4Test, is all that would be required to enable any tests accessing that UI element to run.
Portability Across PlatformsSilkTest uses 4Test, which is an object-oriented, purposebuilt scripting language, to define tests. Many constructs exist within 4Test that are automatically exploited by SilkTest to enable out-of-the-box script portability across platforms, browsers and other variables, such as screen resolution. This means little or no time is spent modifying the tests before running them in different environments or under different operating conditions.
SilkTest Reviewers Guide
Agents for Distributed TestingSilkTest also offers a cost effective way to distribute tests over multiple computers and run them in parallel, thereby shortening the overall test execution cycle. The key driver of this capability is the SilkTest Agent. The Agent is installed on the same machine as the AUT, and drives and monitors the application. In a simple, non-distributed configuration, the SilkTest graphical user interface (GUI), a SilkTest Agent and the AUT all reside on the same machine. In a networked environment, any number of agents can be installed and run on remote machines (controlled by a single SilkTest GUI) in order to distribute testing for concurrent test execution and reduce test cycle time. The ability to have multiple SilkTest Agents controlled by a single SilkTest GUI is necessary in order to correctly test logical transactions that are performed by application components distributed over multiple machines.
ReportingOnce results from a test run are available, SilkTest offers many out-of-the-box mechanisms for results analysis, including charting, graphing, summary reporting, failure and bitmap analysis and updating of expected test results. With extensions to the Recovery System, you can complement the standard results anal