Silk Test

For QTP Documents visit: www.gcreddy.com

Silk TestTHE AUTOMATED TESTING PROCESS

The testing process has these four steps:

Creating a testplan (if you are using QA Organizer) Recording a test frame Creating testcases Running testcases and interpreting their results

Creating a testplan :

If you are using QA Organizer, you begin the automated testing process by creating a testplan. A basic testplan is structured as a hierarchical outline and contains:

Descriptions of individual tests and groups of tests. You can use as many levels of description as you want.

Statements that link the test descriptions in the plan to the 4Test routines, called testcases, that accomplish the actual work of testing. The following figure shows a sample testplan for a search feature. The testplan uses two levels of description: the top level indicates that all the tests are for the Find dialog, and the next level describes eight individual tests.

G.C.Reddy, QTP Trainer, Hyderabad

# 1 #


Recording a test frame

Next, you record a test frame, which contains descriptions, called window declarations, of each of the GUI objects in your application. A window declaration specifies a logical, cross-platform name for a GUI object, called the identifier, and maps the identifier to the object’s actual name, called the tag. In addition, the declaration indicates the type of the object, called its class.

Creating testcases

The 4Test commands in a testcase collectively perform three distinct actions:

Drive the application to the state to be tested. Verify the state (this is the heart of the testcase). Return the application to its original state.

You can either use the powerful object-oriented recorder to automatically capture these 4Test commands as you interact with your application, or you can write the 4Test code yourself if you are comfortable with programming languages. For maximum ease and power, you can combine these two approaches, recording the basic testcase and then extending it using 4Test’s flow of control features.


# 2 #


Running testcases and interpreting results

Next, you run one or more testcases, either by running a collection of scripts, called a suite, or, if you are using QA Organizer, by running specific portions of the testplan. As each testcase runs, statistics are written to a results file. The results file and its associated comparison tools allow you to quickly pinpoint the problems in your application.

Getting started

If you are a first-time user and want to get up and running quickly, you may want to begin with the following:

QuickStart Wizard SilkTest Web Quick Start

Getting started with the QuickStart Wizard

If you are using SilkTest with QA Organizer, you can use the QuickStart Wizard, which greatly simplifies the four steps of automated testing described in the preceding section. When you start SilkTest the first time (or whenever you start and have no open windows), the QuickStart Wizard is displayed automatically. You can also invoke the wizard at any time by selecting File/New and clicking the QuickStart Wizard icon.

To use the QuickStart Wizard, you simply follow its prompts. You can use it to:

Create a testplan.You simply name the file (giving it the .pln extension) and its

directory. Create a test frame, which contains descriptions of the GUI objects in

your application that you want to test.As prompted, you simply open your application and open the

various windows and dialogs that you want to test in the application. The wizard automatically records all the declarations in a file called frame.inc. You don’t have to do any coding.

Record testcases.You name the testcase and provide a description for the testplan,

then simply record the testcase. Again, you don’t have to do any


# 3 #


coding. The wizard automatically saves the testcase in a script (.t) file with the same name as the testplan.

Run testcases.

Example: The following scenario illustrates how you might get started testing the Text Editor application that ships with SilkTest. Since the Text Editor is a standard C application, you don’t need to have any extensions (which provide support for testing applications created with thirdparty development environments) enabled before invoking the wizard. Now you are ready to use the QuickStart Wizard with the Text Editor.

Procedure To use the wizard:

Invoke the wizard by selecting File/New and clicking the QuickStart Wizard icon.

The wizard opens.


# 4 #


Now you will name a new testplan, which will organize and manage your tests.

Click Next.

The following panel displays:


# 5 #


Name the file edit.pln and click Next.

The next step is to record the test frame, which defines all the windows,dialogs, menus, and so on that you want to test.


# 6 #


To create a new test frame, leave New Test Frame selected and click Next.

At this point, the wizard lists all the open (running and not minimized) applications. If Text Editor is not open, you can open it now (it is in the directory where you installed SilkTest). After you open the Text Editor, click on the QuickStart Wizard title bar to see Text Editor added to the list of applications.

Select Text Editor and click Next.

The Capture Windows panel displays, describing the procedure.

Click Next.

Now you simply open a document window and open all the dialogs that you want to test in the Text Editor. When you place the mouse pointer on a window or dialog, the wizard records all the declarations that SilkTest needs in a file called frame.inc in the same directory as your testplan. For example, when you capture the Find dialog, you see this:


# 7 #


When you have finished capturing the windows and dialogs in Text Editor, click Return to Wizard in the Capturing New Windows dialog. Now that you have created your test frame, you are ready to create a testcase.

Click Next twice. You see the following panel.

Name the test FindBox and enter the description “Verify controls in Find dialog.” Click Next. Your test is now being recorded, as indicated by the Record Status window on your screen.


# 8 #


Now go to Text Editor, select Search/Find to open the Find dialog, place your mouse pointer over the dialog’s title bar, and press Ctrl+Alt to verify its state. The Verify Window dialog displays. Click OK to verify all properties for the dialog. Close the Find dialog (to return to your base state), then click Done in the Record Status window. You return to the Wizard and are asked to confirm that the test is what you want.

Click Next.

Run the test by clicking the Run Test Button.

The wizard reports the results. You can move the wizard to the side and look at the results file that is created whenever you run a test.

In the wizard, click Next to save your testcase. The testcase is saved in a script (.t) file with the same name as the testplan (in this case, edit.t).

Click Close to close the wizard. You see a window containing the results file from the test you just ran. In another window is the testplan:

What has been recorded in the testplan

SilkTest has recorded all the needed information for you in the testplan:


# 9 #


The first line records the script file containing the testcase. The second section (under Header:) contains information that the

wizard uses to find files. Click the plus sign to the left of “header” to see the information.

Warning The wizard maintains the information in the header section. You should not make changes in this section.

The last section lists the testcase you just recorded.

Using the wizard from a testplan

Notice the five large buttons at the top of the testplan. Those are wizard buttons. You can click them anytime to have the wizard step you through one of the testing processes.

Whenever you invoke a wizard function by clicking one of its buttons, the wizard tells SilkTest which files to use for this particular testplan. Clicking Disable Wizard removes the buttons from the testplan. You can reactivate the wizard from the Options menu.

How the wizard sets application states

When testing Web applications, the wizard sets the application state to DefaultBaseState, thus activating the recovery system. When testing non- Web applications, the wizard sets the application state to none, thus deactivating the recovery system.


# 10 #


Getting started testing Web applications

If you are using SilkTest to test Web applications, the QuickStart in Testing Web Applications with SilkTest can get you started using a simulated business application on the Web.

The anatomy of a basic testcase

The following figure shows the organization of a typical automated testcase. By convention, you place a group of testcases for a particular application feature into a file called a script.

The testcase keyword Each automated test begins with the testcase keyword. If you have worked with computer languages before, you can consider a testcase a special kind of function, one with built-in error recovery and results reporting capabilities.

The testcase name Immediately after the testcase keyword is the name of the testcase. The testcase is a name of your choice that indicates the testing task being performed.

Object-oriented commands The core of a testcase are the object-oriented 4Test commands that drive, verify, and clean up your application. For example, consider this command from the preceding figure:

TextEditor.File.New.Pick ()


# 11 #


The first part of the command, TextEditor.File.New, is the name of a GUI object. The last part of the command, Pick, is the operation to perform on the GUI object. The dot operator (.) delimits each piece of the command. When this command is executed at runtime, it picks the New menu item from the File Menu of the Text Editor application.

The built-in recovery system

SilkTest maximizes your productivity by providing a completely integrated recovery system, which makes it possible for you to run your tests unattended. When the application you are testing fails, or even crashes, the recovery system restores the application to its base state so that the rest of your tests can continue to run. The recovery system can restore your application to its base state at any point during testcase execution:

Before the first line of your testcase begins running, the recovery system restores the application to the base state even if an unexpected event corrupted the application between testcases.

During a testcase, if an application error occurs, the recovery system terminates the execution of the testcase, writes a message in the error log, and restores the application to the base state before running the next testcase.

After the testcase completes, if the testcase was not able to clean up after itself (for example, it could not close a dialog it opened), the recovery system restores the application to the base state.

SilkTest architecture

Normal use of an application consists of a person manipulating a keyboard and mouse to initiate application operations. The person is said to be interacting with the GUI (Graphical User Interface). During SilkTest testing, SilkTest interacts with the GUI to submit operations to the application automatically. Thus SilkTest can simulate the actions of a person who is exercising all the capabilities of an application and verifying the results of each operation. The simulated user (SilkTest) is said to be driving the application. The application under test reacts to the simulated user exactly as it would react to a human user. SilkTest consists of two distinct software components that execute in separate processes:

The SilkTest host software The 4Test Agent software


# 12 #


SilkTest host software The SilkTest host software is the program you use to develop, edit, compile, run, and debug your 4Test scripts and testplans. This manual refers to the system that runs this program as the host machine or the SilkTest machine.

The Agent The 4Test Agent is the software process that translates the commands in your 4Test scripts into GUI-specific commands. In other words, it is the Agent that actually drives and monitors the application you are testing. One Agent can run locally on the host machine. In a networked environment, any number of Agents can run on remote machines. This manual refers to the systems that run remote Agents as target machines.

In a client/server environment, SilkTest drives the client application by means of an Agent process running on each application’s machine. The application then drives the server just as it always does. SilkTest is also capable of driving the GUI belonging to a server or of directly driving a server database by running scripts that submit SQL statements to the database. These methods of directly manipulating the server application are intended to support testing in which the client application drives the server.

The Agent menu If the Agent has been started, an Agent button appears in the taskbar. Press the right mouse button over the Agent button to display the Agent menu. The menu contains the standard menu items for a Window, plus the following:


# 13 #


The 4Test language

4Test is an object-oriented fourth-generation language (4GL) designed specifically with the needs of the QA professional in mind. 4Test’s powerful features are organized into three basic kinds of functionality:

A robust library of object-oriented classes and methods that specify how a testcase can interact with an application’s GUI objects.


# 14 #


A set of statements, operators, and data types that you use to add structure and logic to a recorded testcase.

Object-oriented features

Classes and methods Classes are the core of object-oriented languages. For each kind of GUI object, there is an associated class that defines the actions, called methods, that can be performed on all objects of that type. For example, the PushButton class defines the methods that can be performed on all the pushbuttons in your application.

Inheritance Classes are organized in a hierarchy. The reason for this is that classes are related to each other, sharing characteristics common to their parent classes. For example, the parent class Control defines all the characteristics common to all kinds of controls: check boxes, text fields, pushbuttons, and so on. In this way, each class does not need to define all the methods it needs; the class can just inherit the existing definition from its ancestor classes. For example, one action you can perform on objects of class DialogBox is the GetDefaultButton method. This method returns the identifier of the default pushbutton, which is the pushbutton that is pressed when the Return or Enter key is pressed. You could not use this method with objects that were not dialog boxes, such as menus or text fields. When you record testcases, the proper 4Test methods for each of your manual actions are recorded automatically for you, so it really is transparent to you which class the recorded methods belong to. However, if you decide to write or augment a testcase by hand, you can look up the class and the methods it supports in the online Help or Library Browser.

The class hierarchy The following illustration shows the hierarchy of the core classes of GUI objects recognized by SilkTest.

Note Extensions to SilkTest provide additional classes that are specific to particular development environments. For example, the Java extension includes classes that are specific to GUI objects in Java applications.

Properties A property is a characteristic of an object that you can access directly. You typically verify a GUI object’s properties in a testcase.


# 15 #


StatementsBy using 4Test flow-of-control statements, you can add logic and robustness to a recorded testcase. The following table summarizes the statements.


# 16 #


Example The following 4Test code shows how to use the if statement to conditionally execute a method. The code picks the Close menu item from


# 17 #


the File menu of the Text Editor application, which causes a message box to appear if there is unsaved work. The if statement tests for the existence of the message box.

TextEditor.File.Close.Pick ()if MessageBox.Exists () // if the message box existsMessageBox.No.Click () // then dismiss it

Data types and variables

Built-in data types 4Test provides the following built-in data types:

ANYTYPELIST ARRAY BOOLEAN BROWSERTYPEDATACLASS DATATYPE DATE DATETIMEFONTSTYLE GUITYPE HANDLE INTEGERLIST LONG NUMBER REALSEMAPHORE SET STRING TABLECOLTABLEROW TIME WINDOW

C data types for DLL functions

In addition to the 4Test data types, the following C data types are supportedfor use in calling functions in DLLs.

Char int short long floatunsigned char unsigned int unsigned short unsigned long double

The first two columns above show data types that correspond to the 4Test INTEGER data type. The third column corresponds to the REAL data type.

User-defined types You can also create new data types, including enumerated types and records. For example:

type FILE is LIST OF STRINGtype COLOR is enumredgreen

Adding variables to a testcase


# 18 #


In the following testcase a record data type is defined that contains each of the data values the testcase needs. The methods in the testcase process fields from the record instead of literal data values.

type SEARCHINFO is record

STRING sText // Text to enter in document windowSTRING sPos // The starting position of searchSTRING sPattern // The string to search forBOOLEAN bCase // Case-sensitive or notSTRING sDirection // The direction of the searchSTRING sExpected // The string you expect to findtestcase Find (SEARCHINFO Data)TextEditor.File.New.Pick ()DocumentWindow.Document.TypeKeys (Data.sText + Data.sPos)TextEditor.Search.Find.Pick ()Find.FindWhat.SetText (Data.sPattern)Find.CaseSensitive.SetState (Data.bCase)Find.Direction.Select (Data.sDirection)Find.FindNext.Click ()Find.Cancel.Click ()DocumentWindow.Document.VerifySelText ({Data.sExpected})TextEditor.File.Close.Pick ()MessageBox.No.Click ()

Built-in functions

4Test contains a function library to handle the most common programming tasks. The following table summarizes the functions by category:

Application state Array manipulation Char/string conversionData type manipulation Distributed processing Exception handlingFile manipulation List manipulation Numeric operationsRandom values Results file operations Script informationSemaphore operations Set manipulation StartupString manipulation System calls Timers, time/date infoWindow information

The basic toolsYou can use the following powerful SilkTest tools to manage, execute,

and


# 19 #


interpret your tests, including:

The outline editor The results processor The debugger

The outline editor

You interact with testplans, testcases, and results files in the outline editor, an easy-to-use interactive environment that is consistent throughout all phases of testing. Because these files contain large amounts of information, a structured, hierarchical outline provides an ideal organizational model. For example, the following figure shows a testcase in the outline editor:

Notice how the statements that make up the body of the expanded testcase are indented. The outline editor uses indentation to differentiate levels. A minus icon ( ) indicates that a level is fully expanded. Similarly, a plus icon ( ) indicates that a level is collapsed and has hidden detail. Click on the icons to expand and collapse the outline, to show the desired level of detail.

Outlining commands As you work in the outline editor, you can use menu, keyboard, or tool bar commands to change indentation levels. The following table summarizes thecommands:


# 20 #


Line continuation character

Use Shift+Enter to force a soft line break within a long line of code. The outline editor considers such lines as one statement.

Using an ASCII text editor

Typically, you write scripts and testplans using the built-in outline editor, which is optimized for managing scripts and testplans.

However, you can also use your favorite ASCII text editor to write and edit scripts and testplans, then open them in the outline editor for execution. You can even make changes in the outline editor, save the changes, then reopen the modified file in your ASCII editor for continued editing.

Procedure To write and edit scripts and testplans in an ASCII editor:

Follow these rules when creating files in your ASCII editor:

Begin each line with [ ] (left bracket, followed by a space, followed by a right bracket).

Use tabbing to indent levels. If you want to use spaces, instead of tabs, to denote levels, include the

following line at the top of the file:

[t n] where n is the number of spaces that correspond to a tab. For example, if [t4] is the first line of a file, every 4 spaces will be converted into a tab.

Save the file using the appropriate file extension.


# 21 #


You can use these scripts and testplans the same way you use files created in the outline editor: Select File/Open to open them. You can make and save changes in the outline editor, then reopen the file in your ASCII editor for further revision. Using Microsoft Word You can also use Microsoft Word to develop testplans.

The results processor

After a testcase runs, a results file is displayed, as shown in the following figure. Use the commands on the Results menu to process results.

If a testcase encountered an error, click on the plus icon ( ) preceding the test description to reveal a brief description of the error. To invoke


# 22 #


comparison tools that let you see just how the results differ from the baselines, click on the box icon, as shown in this figure:

The debugger

SilkTest comes with a powerful set of debugging tools to help you find problems with your scripts. Using these debugging tools, you can step through a script a line at a time and stop at specified breakpoints, as well as examine local and global variables and enter expressions to evaluate. The following figure shows a testcase in the debugger with a breakpoint set.


# 23 #


CREATING TESTPLANS

If you are using QA Organizer, the first step in creating automated tests is to create a testplan. (If you are not using QA Organizer, the first step is creating a test frame. A QA Organizer testplan consists of two distinct parts: an outline that describes the test requirements, and statements that connect the outline to the 4Test scripts and testcases that implement the test requirements.

Starting a new testplan

Procedure To start a new testplan:

Select File/New.

Select Testplan and click OK. An empty testplan window opens.


# 24 #


Understanding the outline model

Because a testplan is made up of a large amount of information, a structured, hierarchical outline provides an ideal model for organizing and developing the details of the plan.

Sample outline for a word search featureFor example, consider the Find dialog from the Text Editor application,

which allows a user to search in a document:

A user enters the character(s) to search for in the Find What text field, checks the Case Sensitive check box to consider case, and selects either the Up or Down radio button to indicate the direction of the search. Here is what a partial outline of the testing requirements might look like:

I. Find dialogA. Case-sensitive

1. Forward searcha. Search for a characterb. Search for a word

2. Backward searcha. Search for a characterb. Search for a word

B. Case-insensitive1. Forward search

a. Search for a characterb. Search for a word

2. Backward searcha. Search for a characterb. Search for a word

This outline describes eight distinct tests:


# 25 #


1. Find dialog, case sensitive, forward, character search2. Find dialog, case sensitive, forward, word search3. Find dialog, case sensitive, backward, character search4. Find dialog, case sensitive, backward, word search5. Find dialog, case insensitive, forward, character search6. Find dialog, case insensitive, forward, word search7. Find dialog, case insensitive, backward, character search8. Find dialog, case insensitive, backward, word search

Concepts and terminology

Notice that the full description for each of these eight tests is formed by joining the descriptions along the branch that leads to the test. This can be seen more clearly when the outline is rewritten as a tree:

Test Descriptions Because the last, or terminal, point along a path completes a test description, each terminal point in an outline is referred to as a test description. When QA Organizer is interpreting your outlines, it knows that when it reaches a terminal point, there should be a test that can be executed. Test descriptions by default are displayed in blue in the outline editor.

Group Descriptions The other descriptive lines in an outline are called group descriptions, because they describe a group of tests, not a single test. For example, in the sample plan shown above, the single descriptive line Find dialog describes a group of eight tests and the descriptive line Case Sensitive describes a group of four tests. Group descriptions by default are displayed in black.Scope In addition to test descriptions and group descriptions, a testplan also contains the QA Organizer statements that implement the test requirements.


# 26 #


The scope of a statement refers to the portion of the testplan where the statement applies. A statement placed at the group description level applies to all the test descriptions contained by the group. Conversely, a statement placed at the test description level applies only to that test description. Levels in QA Organizer are represented by indentation, and you change the level of a line by using the Outline/Move Left and the Outline/Move Rightcommands, both of which have keyboard, tool bar, and popup menu shortcuts.

Benefits There are four significant benefits to structuring a testplan as a hierarchicaloutline. An outline:

Assists the testplan author in developing thoughts about the test problem, by promoting and supporting a top-down approach to test planning

Yields a comprehensive inventory of test requirements, from the most general, through finer and finer levels of detail, to the most specific

Allows the QA Organizer statements that actually implement the tests to be shared by group descriptions or used by just a single test description

Provides reviewers with a framework for evaluating the thoroughness of the plan and for following the logic of the testplan author

Linking the testplan to scripts and testcases

Once you have a descriptive outline of the test requirements, you can associate the outline with the 4Test scripts and testcases that implement the requirements. You create this association by inserting script and testcase statements in the appropriate locations in the outline.

Where to insert script and testcase statements

You can insert a script and testcase statement for each test description, although placing a statement at the group level when possible eliminates redundancy in the testplan. For example, since it is usually good practice to place all the testcases for a given application feature into a single script file, you can reduce the redundancy in the testplan by specifying the script statement at the group level that describes that feature.The following testplan illustrates this, specifying the script find.t once for the entire group of tests of the Find dialog (the script statement is highlighted for emphasis):


# 27 #


Three ways to link a testplan with a script or testcase

There are three ways to associate a testplan with a 4Test script and testcase:

Use the Testplan Detail dialog to automate the process Enter the script and testcase statements manually Record the testcase from within the testplan

How to link using the Testplan Detail dialog

The Testplan Detail dialog automates the process of linking to scripts and testcases. It lets you browse directories and select script and testcase names, and it enters the correct QA Organizer syntax into the plan for you.

Procedure To use the Testplan Detail dialog to link a test description or agroup description to a script or testcase:

Place the insertion cursor on either a test description or a group description.

For example, the following figure shows how to position the cursor to link the first test description:


# 28 #


Select Testplan/Detail. QA Organizer invokes the Testplan Detail dialog, with the Test Execution tab showing:

The multi-line list box at the top of the dialog displays the line in the testplan that the cursor was on when the dialog was invoked, as well as its ancestor lines. The black arrow icon indicates the current line. The current line appears in black and white, and the preceding lines appear in blue.


# 29 #


If you know the names of the script and testcase, enter them in the Script and Testcase fields, respectively, and skip to step 8.

If you are unsure of the script name, click the Scripts pushbutton to the right of the Script field to browse for the script file. QA Organizer invokes the Testplan Detail - Script dialog:

Navigate to the appropriate directory and select a script name by doubleclicking on it or by selecting it and clicking the OK pushbutton.

The Testplan Detail Script dialog closes and the script name is entered in the Script field. For example:


# 30 #


Click the Testcase pushbutton to the right of the Testcase field, to browse for the testcase name: QA Organizer invokes the Testplan Detail - Testcase dialog:

This dialog shows the names of the testcases that are contained in the selected script. Testcases are listed alphabetically, not in the order in which they occur in the script.

Select a testcase from the list, by double-clicking on it or by highlighting it and clicking the OK pushbutton. The Testplan Detail Testcase dialog closes and the testcase name is entered in the Testcase field. For example:

Click OK. The script and testcase statements are entered in the plan. For example:


# 31 #


You are done.

How to link manually If you feel comfortable with the syntax of QA Organizer statements and know the locations of the appropriate script and testcase, you can enter the script and testcase statements manually.

Procedure To manually link a description to a script or testcase:

Place the insertion cursor at the end of a test or group description, press the Enter key to open up a new line, and indent the new line one level.


# 32 #


Enter the script and/or testcase statements using the following syntax, substituting actual names for myscript and MyTestcase:

script: myscript.ttestcase: MyTestcaseFor example, to link to the find.t script:


# 33 #


If you enter a statement correctly, it appears in red, the default color used for QA Organizer statements. If not, it will either appear in blue, indicating the line is being interpreted as a test description, or black, indicating it is being interpreted as a group description.

Documenting manual tests in the testplan

Your QA department might do some of its testing manually. You can document the manual testing in the testplan. In this way, the planning, organization, and reporting of all your testing can be centralized in one place. To indicate that a test description in the testplan is implemented with a manual test, use the value manual in the testcase statement, as in:

testcase: manual.

For example,

the following testplan indicates that the case-insensitive tests are manual:


# 34 #


Tracking manual tests You can describe the state of each of your manual tests in a dialog in SilkTest. This information is used in reports.

Procedure To describe the state of a manual test:

Open a testplan containing manual tests. Select Testplan/Manual Tests.

The Update Manual Tests dialog is displayed. It lists all manual tests in the current testplan.


# 35 #


Select a manual test and document it.


# 36 #


RECORDING A TEST FRAME

Introduction Before you begin recording testcases, you first record a test frame for your application. The test frame is the backbone that supports your testcases and scripts. It is a file that contains all the information about your application’s GUI objects that SilkTest needs when you record testcases. This information minimally consists of a declaration for each GUI object, but can also include any data that you want to associate with each GUI object, as well as any new classes and methods that you want to define.

Why window declarations make your tests robust

Window declarations are extremely powerful features, for two reasons.


# 37 #


Declarations specify logical names

A window declaration specifies a cross-platform, logical name for a GUI object, called the identifier, and maps the identifier to the object’s actual name, called the tag. Because your testcases use logical names, if the object’s actual name changes on the current GUI, on another GUI, or in a localized version of the application, you need only change the tag in the window declarations; you don’t need to change any of your scripts.

Declarations can encapsulate data and functions

You can add variables, functions, methods, and properties to the basic window declarations recorded by SilkTest. For example, you can add variables to a dialog box declaration that specify what the tab sequence is, what the initial values are, and so on. You access the values of variables at runtime as you would a field in a record.

Preparing to record window declarations

If you plan to test Web applications, make sure you enable the correct browser extension(s) on your target and host machine. If you do not plan to test Web applications, you must disable all browser extensions on the host machine.

Enabling browser extensions Enabling browser support on a target machine

By default, SilkTest enables all the browser extensions on your target machine during the installation procedure. To change the default settings or verify your current settings, you must invoke a utility called the Extension Enabler, as described in the following procedure.

Note If you are running local tests—that is, your target and host are the same machine—you must still ensure that browser extensions are enabled appropriately by running the Extension Enabler on the host machine.

Procedure To enable support for browsers on a target machine:

From the Silk program group, click Extension Enabler.

The Extension Enabler dialog opens, as in this example:


# 38 #


Enable support for the browsers you want to use, if they aren’t already enabled. In the Primary Extension field for a browser, choose Enabled.

Select any other extensions that the browser needs for testing such as ActiveX or fault trapping. We recommend that you do not turn on fault trapping until you really need it.

Click OK to close the Extension Enabler dialog.

En abling browser support on a host machine

You enable support for browsers on the host machine using the Extensions dialog. Be advised that there is overhead associated with having more than one browser extension enabled, so do so only if you are actually testing more than one browser in an automated session.

Procedure To enable support for browsers on a host machine:

Start SilkTest. Select Options/Extensions. The Extensions dialog opens. All the

installed extensions are listed, including the browser extensions, as in this example:

Enable the browser extension(s) you want to use. In the Primary Extension field for a browser, choose Enabled. Disable any browser


# 39 #


extensions you do not plan to use by choosing Disabled in the Primary Extension field and unchecking their other extensions.

Select any other extensions that the browser needs for testing such as ActiveX or fault trapping. We recommend that you do not turn on fault trapping until you really need it.

Note If you enable support for ActiveX in this dialog, you must make sure that it is enabled in the Extension Enabler dialog as well.

Click OK to close the Extensions dialog. You can get information about the files used by an extension by selecting an extension and clicking the Details pushbutton.

Disabling browser extensions

If you testing non-Web applications, you must disable browser extensions on your host machine. This is because the recovery system works differently when testing Web applications than when testing non-Web applications.

Procedure To disable support for browsers on a host machine:

Start SilkTest. Select Options/Extensions. The Extensions dialog opens. Disable any browser extens ions you do not plan to use by choosing

Disabled in the Primary Extension field and unchecking all check boxes in the Other Extensions columns.

Click OK to close the Extensions dialog.

How to record window declarations

Two stages Record window declarations for your application in two stages:

Record the window declarations for the main window (including its menus and controls).

Bring up each dialog one at a time and record a declaration for each.

Recording the main window and menu declarations

The following discussion pertains to non-Web applications.

Procedure To record declarations for the main window and menu hierarchy of your application:


# 40 #


Start up your application. Select File/New. The New dialog appears. Select the Test Frame radio button and click OK.

The New Test Frame dialog is displayed, allowing you to create a test frame file for an application displayed in the Application list box. The Application list box displays all applications that are open and not minimized; if your test application is not listed, click Cancel, open your application, and select File/New again.

The default file name for the new test frame file is frame.inc. (The .inc extension denotes an include file, a file that stores declarations.) By default, subsequent frame files in the same directory are named frame1.inc, frame2.inc, and so on. You can edit the path, file name, or both. Click the Browse pushbutton if you need help editing the name or path.

Select your application from the Application list box. The Command Line, Working Directory, and 4Test Identifier fields are updated for your application:

The Command Line field displays the path of the application’s executable.

The Working Directory field names the directory in which the application’s program files and documents are located.


# 41 #


The 4Test Identifier field displays the default identifier of the application’s main window, which is based on the actual caption of the main window.

Click OK.

The new test frame file is created. The file contains the 4Test declarations for the main window and all its menus, as well as a generic declaration that is valid for each of the standard message boxes in the application. For example:

Window declarations appear in the outline editor, which means that the declarations for individual GUI objects can be expanded to show detail, collapsed to hide detail, and edited if necessary.

File name is stored in runtime options

When you record a test frame, the full path of the test frame file is added to the Use Files field of the Runtime Options dialog. This means that SilkTest can use of the information contained in the declarations and recognize the GUI objects in your application when you record testcases.

Recording declarations for a dialog

After you record your test application’s main window and menus, you record all the dialogs you want to test.

Procedure To record your application’s dialogs, use this procedure once for each dialog in your application:

Make sure that the test frame file that contains the declarations for the application’s main window is open. The dialog declarations will be appended to this file.

Select Record/Window Declarations. The Record Window Declarations dialog is displayed.


# 42 #


Make your application active and invoke one of its dialogs, referred to in this procedure as the target dialog.

If necessary, arrange windows so that you can see the target dialog. Position the cursor on the title bar of the target dialog. Note that as

you move the cursor toward the title bar, the contents of the Window Declaration list box change dynamically to reflect the object at which you’re pointing, as well as any contained objects. When the cursor is positioned correctly, the Window Detail group box (upper left) shows the caption of the dialog in the Identifier field.

Press Ctrl+Alt.

The declaration is frozen in the lower half of the dialog.

Close the target dialog. Click on the Paste to Editor pushbutton. The information in the Record

Window Declarations dialog is cleared, and the newly recorded declarations are appended to the test frame after the last recorded declaration.

If finished recording declarations, click the Close pushbutton on the Record Window Declarations dialog. Otherwise, press the Resume Tracking pushbutton to begin recording the declarations for another dialog.


# 43 #


Recording a login window

Many commercial applications begin with a login window, which is not accounted for when you record the test frame. Therefore, make sure that you invoke this window and record a declaration for it when you are recording the declarations for your application’s dialogs.

The wStartup constant When you record the test frame, a constant called wStartup is created. By assigning the identifier of the login window to wStartup and by recording a new invoke method, your tests can start the application, enter any required information into the login window, then dismiss the login window

Saving the test frame After you finish recording declarations for each one of your application’s dialogs, save the test frame file.

Procedure To save a test frame, select File/Save when the test frame is the active window. If it is a new file, it is automatically named frame.inc (if you already have a frame.inc file, a number is appended to the file name). You can use File/Save As to select another name. When saving a file, SilkTest does the following:

Saves a source file, giving it the .inc extension; the source file is an ASCII text file, which you can edit

Saves an object file, giving it the .ino extension; the object file is a binary file that is executable, but not readable by you

Example If you name a frame file myframe and save it, you will end up with two files: the source file myframe.inc, in the location you specify, and the object file myframe.ino.

DESIGNING AND RECORDING TESTCASES

Introduction This chapter shows you how to use the powerful SilkTest recorder to automatically capture the actions you perform as you interact with your application, creating an automated testcase. The cornerstone of an automated test is the verification stage, in which the test verifies that the state of the application matches the expected (baseline) state. Using the recorder, you can record object-appropriate verification of your application’s state, data, or appearance.


# 44 #


Test running the built-in recovery system

Before you begin to design and record testcases, you should first make sure that the built-in recovery system can close representative dialogs from your application. Although the recovery system is robust enough to be able to close almost any application window, some applications may have windows that close in an unconventional fashion.

Three types of dialogs should be tested

Here are the three kinds of dialogs you should test: A modal dialog (a dialog that locks you out of the rest of your

application until you dismiss it) A non-modal dialog A non-modal dialog that causes the display of a confirmation dialog

Procedure To test the recovery system’s ability to close your application’s dialogs:

Start SilkTest. If you have not already done so, record a test frame for your

application. Make sure your application’s test frame file is listed in the Use Files

field in the Runtime Options dialog (select Options/Runtime). Start your application and invoke a representative dialog. Select Run/Application State.

The Run Application State dialog is displayed:

Select the DefaultBaseState application state and click Run.


# 45 #


The DefaultBaseState routine is executed, which should close the dialog and any open windows, then display a results file. For example, the following figure shows the results file produced when the recovery system closes the Text Editor’s Open dialog:

If the built-in recovery system cannot close one of the three representative dialogs, you need to modify the recovery system so that it understands how to close the dialog.

Testcase design principles

This section explains the methodology you use when you design and record a testcase.

Two kinds of testcases

There are two basic types of testcases, each with its own purpose.

Level 1 tests, often called smoke tests or object tests, verify that an application’s GUI objects function properly. For example, they verify that text fields can accept keystrokes and check boxes can display a check mark.

Level 2 tests verify an application feature. For example, they verify that an application’s searching capability can correctly find different types of search patterns. You typically run Level 1 tests when you receive a new build of your application, and do not run Level 2 tests until your Level 1 tests achieve a specific pass/fail ratio. The reason for this is that unless your application’s graphical user interface works, you cannot actually test the application itself.

A testcase has three stages


# 46 #


Each testcase that you record should have the following three stages.

It drives the application from the initial state to the state you want to test.

It verifies that the actual state matches the expected (correct) state. (Your QA department might use the term baseline to refer to this expected state.) This stage is the heart of the testcase.

It cleans up the application, in preparation for the next testcase, by undoing the steps performed in stage 1.

Each testcase is independent

Each testcase you record should perform its own setup in stage 1, and should undo this setup in stage 3, so that the testcase can be executed independently of every other testcase. In other words, the testcase should not rely upon the successful or unsuccessful completion of another testcase, and the order in which it is executed should have no bearing on its outcome. If a testcase relies on a prior testcase to perform some setup actions, and an error causes the setup to fail or, worse yet, the application to crash, all subsequent testcases will fail because they cannot achieve the state where the test is designed to begin.

A testcase has a single purpose

Each testcase you record should verify a single aspect of the application in stage 2. When a testcase designed in this manner passes or fails, it’s easy to determine specifically what aspect of the target application is either working or not working. If a testcase contains more than one objective, many outcomes are possible. Therefore, an exception may not point specifically to a single failure in the software under test but rather to several related function points. This makes debugging more difficult and time-consuming and leads to confusion in interpreting and quantifying results. The net result is an overall lack of confidence in any statistics that might be generated.

A testcase starts from a base state

In order for a testcase to be able to function properly, the application must be in a stable state when the testcase begins to execute. This stable state is called the base state. The recovery system is responsible for maintaining the base state in the event the application fails or crashes, either during a testcase’s execution or between testcases.


# 47 #


DefaultBaseState To restore the application to the base state, the recovery system contains a routine called DefaultBaseState that makes sure that

The application is running and is not minimized All other windows (for example, dialogs) are closed The main window of the application is active

Defining test requirements

When defining test requirements, the goal is to rigorously test each application feature. To do so, you need to decide which set of inputs to a feature will provide the most meaningful test results.

Example: a word processor’s search feature

For purposes of illustration, this section develops test requirements for the searching feature of the sample Text Editor application. This feature uses the Find dialog.

When a user enters the criteria for the search and clicks the Find Next pushbutton, the searching feature attempts to locate the string. If the string is found, it is selected (highlighted). Otherwise, an informational message is displayed.

What data does the eature expect

A user can enter three pieces of information in the Find dialog:

The search can be case sensitive or insensitive, depending on whether the Case Sensitive check box is set or unset.

The search can be forward or backward, depending on whether the Down or Up radio button is selected.

The search can be for any combination of characters, depending on the value entered in the Find What text field.

Create meaningful data combinations


# 48 #


To organize this information, it is helpful to construct a table that lists the possible combinations of inputs. From this list, you can then decide which combinations are meaningful and should be tested. A partial table for the Find dialog is shown below:

Overview of recording the stages of a testcase

As explained in “Testcase design principles”, a testcase performs its actions in three stages. The following table illustrates these stages, describing in high-level terms the steps for each stage of a sample testcase that tests whether the Find facility is working.


# 49 #


Preparing to record

Procedure To invoke the Record Testcase dialog and set up the test:

Select Record/Testcase. If a script file is not the active window, you are prompted for a

file name.


# 50 #


If prompted, specify the name of either a new or an existing script file and click OK. The Record Testcase dialog appears.

Type the name of your testcase in the Testcase Name field of the Record Testcase dialog. By default the name is Test1 (unless that name is already used in the current script file, in which case the default name is Test2, and so on). Testcase names are not case sensitive; they can be any length and consist of any combination of alphabetic characters, numerals, and underscore characters.

Select DefaultBaseState in the Application State field to have the built-in recovery system restore the default base state before the testcase begins executing. Select another application state if you want the recovery system to perform an additional sequence of steps after the default base state is restored but before the testcase begins executing. The application state you chose is made the default application state and is suggested in the Record Testcase dialog the next time you record a testcase.

If you do not want SilkTest to display the status window it normally shows during playback when driving the application to the specified base state—perhaps because the status bar obscures a critical control in the application you are testing—uncheck the Show AppState status window check box.

Click the Start Recording pushbutton. The following sequence of events occurs:


# 51 #


The Record Testcase dialog closes. Your application is started, if it was not already running. The editor window disappears from the display. The Record Status window appears.

Using the Record Status window

As you move around the application while recording, the Record Status window dynamically updates to show the object that the recorder thinks the mouse is currently over or the control that has focus.

Use this information to make sure that the recorder has kept up with your actions (sometimes the recorder can fall behind if your system’s resources are low). In particular, check the Record Status window before pressing Ctrl+Alt to do your verification in order to make sure you are verifying the correct object.

Recording the setup stage

Procedure To record the setup stage:

Interact with your application, driving it to the state that you want to test. As you interact with your application, your interactions are recorded in the Testcase Code field of the Record Testcase dialog, which is not visible.

To review what you have recorded, click the Done button in the Record Status window. The Record Testcase dialog is redisplayed, containing the 4Test code that has been recorded for you. To resume recording your interactions, click the Resume Recording button in the dialog.

To temporarily suspend recording, click the Pause Recording button on the Record Status window. You might want to halt recording to leave your desk or to prevent some interactions from being recorded.

Recording the verification stage


# 52 #


Testing applications involves verifying that the state of a GUI object is the same as the baseline state that you expect. You can:

Test the characteristics of an object Capture all or part of a bitmap to verify the appearance of an object Use a method to verify an object

Procedure To record the verification stage:

To add a verification statement, position the mouse cursor over the object. Look at the Record Status window and make sure it is listing the object you want to verify. If so, press Ctrl+Alt. (On OSF/Motif, make the window you want to verify active and press Ctrl+Alt. How you make a window active on Motif depends on how you set the keyboardFocusPolicy resource.) The Verify Window dialog appears over your application window.

The Window field, in the top-left corner of the dialog, displays the name of the object you were pointing at when you pressed Ctrl+Alt.

If the name in the Window field is incorrect, press Cancel to close the dialog and return to the application. Point to the object you want to


# 53 #


verify and press Ctrl+Alt again. The Verify Window dialog has three tabs. The dialog has these tabs by default.

Attribute tab On non-Windows platforms, the Verify Window dialog has an Attribute tab instead of a Property tab. You use the Attribute tab on non- Windows platforms to verify a characteristic of the object.

About attributes and properties Properties and attributes in this context are similar—they both are used to verify a characteristic of the object. However, properties are more encompassing, more flexible, and easier to use. For example, using attributes you can only verify one attribute at a time (or verify every attribute for an object and all its children); using properties you can verify selected properties of an object and any or all of its children at the same time. Under Windows, you can choose to verify an object using attributes instead of properties, though you will seldom choose attributes, except for compatibility with previous releases of QA Partner.

Procedure To verify using attributes under Windows:

Select Options/Recorder. The Recorder Options dialog appears. Deselect the Verify Using Properties check box.

Note You will not be able to deselect this check box if you have enabled enhanced support for Visual Basic or PowerBuilder, both of which require properties for verification.

Click OK.


# 54 #


With Verify Using Properties deselected, the next time you go to verify an object, the Verify Window dialog will have an Attribute tab, instead of a Property tab.

Verifying using properties

You will perform most of your verifications using properties. Each object has many characteristics, or properties. For example, as shown in the following illustration, dialog boxes have these verification properties:

Caption, Children, DefaultButton, Enabled, Focus, Rect, and State.

Caption is the text that appears in the dialog box’s title bar; Children is a list of all the objects contained in the dialog box, DefaultButton is the button that is invoked when you press Enter, and so on. In your testcases, you can verify the state of any of these properties. You can also, in the same testcase, verify properties of children of the selected object. In the preceding illustration, notice that the child objects in the Find dialog box (such as the text field FindWhat and the check box CaseSensitive) are also selected for verification. By recording verification statements for the values of one or


# 55 #


more of an object’s properties, you can determine whether the state of the application is correct or in error when you run your testcases.

Verifying an object’s bitmap

A bitmap is a picture of some portion of your application. Verifying a bitmap is usually only useful when the actual appearance of an object needs to be verified to validate application correctness. For example, if you are testing a drawing or CAD/CAM package, a testcase might produce an illustration in a drawing region which you want to compare to a baseline. Other possibilities include the verification of fonts, color charts, and certain custom objects.

Bitmap tips When comparing bitmaps, keep the following in mind:

Bitmaps are not portable between GUIs. The formats are:

A bitmap comparison will fail if the image being verified does not have the same screen resolution, color, window frame width, and window position when the testcase is run on a different machine than the one on which the baseline image was captured.

Make sure that your testcase sets the size of the application window to the same size it was when the baseline bitmap was captured.

Capture the smallest possible region of the image so that your test is comparing only what is relevant.

If practical, do not include the window’s frame (border), since this may have different colors and/or fonts in different environments.

Procedure To verify the GUI object’s appearance using a bitmap:

Select Record/Testcase to begin recording a testcase (or select Record/Actions if you want to record a verification statement in an


# 56 #


existing testcase) and drive your application to the state you want to verify.

When you are ready to record a verification statement, position the mouse cursor over the object you want to verify, then press Ctrl+Alt. The Verify Window dialog appears over your application window.

Display the Bitmap tab. Select the region to update: Entire Window, Client Area of Window

(that is, without scroll bar or title bar), or Portion of Window. In the Bitmap File Name field, enter the full path of the bitmap file that

will be created. The default path is based on the current directory. The default file name for the first bitmap is bitmap.bmp (on OSF/Motif, it is bitmap.xpm). Use the Browse pushbutton if you need help choosing a new path or name.

Click OK. If you selected Entire Window or Client Area of Window, the bitmap is captured and you return to your test application. If you select Portion of Window, position the cursor at the desired location to begin capturing a bitmap. While you press and hold the left mouse button, drag the mouse to the screen location where you want to end the capture. Release the mouse button.

Complete your testcase.

Bitmap Agent options Various bitmap options determine bitmap verification behavior:


# 57 #


To globally set these options for a group of scripts, use the Agent Options dialog. To locally set these options within a script or testcase, use the SetOption method, for example:

Agent.SetOption (OPT_BITMAP_MATCH_COUNT, 3)

Verifying using methods

Each class has a set of methods associated with it, including built-in verification methods. You can verify an object’s state using one of these built-in verification methods or by using other methods in combination with the built-in Verify function.

Using a verification method

A class’s verification methods always begin with Verify. For example, a TextField has the following verification methods; VerifyPosition, VerifySelRange, VerifySelText, and VerifyValue.Procedure To verify an object using a verification method:

Select Record/Testcase to begin recording a testcase (or select Record/Actions if you want to record a verification statement in an existing testcase) and drive your application to the state you want to verify.

When you are ready to record a verification statement, position the mouse cursor over the object you want to verify, and press Ctrl+Alt. The Verify Window dialog appears over your application window.

Display the Method tab. The methods for the selected class are listed on the left. Select the Include Inherited check box to see methods that the class inherits.

Select the verification method from the list. If the method takes arguments, they are displayed.

Fill in any arguments as needed and click OK. Complete your testcase.

For example, here is a testcase that verifies that the text in the TextField Replace.FindWhat is “myText.” It uses the built-in verification method VerifyValue.

testcase VerifyMethodTest ()TextEditor.Search.Replace.Pick ()Replace.FindWhat.VerifyValue ("myText")Replace.Cancel.Click ()


# 58 #


Using the Verify function

You can use the built-in Verify function to verify that two values are equal and generate an exception if they are not. Typically, you use Verify to test something that doesn’t map directly to a built-in property or method. Verify has the following syntax:

Verify (aActual, aExpected [, sDesc])

aActual The value to verify. ANYTYPE.aExpected The expected value. ANYTYPE.sDesc Optional. A message describing the comparison. STRING.Usually, the value to verify is obtained by calling a method for the objectbeing verified; you can use any method that returns a value.

Procedure To verify an object using the Verify function:

Select Record/Testcase to begin recording a testcase (or select Record/ Actions if you want to record a verification statement in an existing testcase) and drive your application to the state you want to verify.

When you are ready to record a verification statement, position the mouse cursor over the object you want to verify, and press Ctrl+Alt. The Verify Window dialog appears over your application window.

Display the Method tab. The methods for the selected class are listed on the left. Select the Include Inherited check box to see methods that the class inherits.

Select the method that will return the expected value; provide any needed arguments. You can specify a built-in method or a user-defined method (as long as it returns a value).

Click OK. You return to the test application. Complete your testcase. Paste the testcase into your script. In the editor, wrap the Verify function around the method that returns

the expected value as follows: Make the method call the first argument, specify the expected value as the second argument, and provide an error message string optionally as the third argument.

See the following for an example.

Example Say you want to verify the number of radio buttons in the Direction RadioList in the Replace dialog in the Text Editor. There is no property or


# 59 #


method you can directly use to verify this. But there is a method for RadioList, GetItemCount, which returns the number of radio buttons. You can use the method to provide the actual value, then specify the expected value in the script, as follows:When doing the verification, position the mouse pointer over the RadioList and press Ctrl+Alt. Select the Method tab in the Verify Window dialog, and select the GetItemCount method. Click OK to close the Verify Window dialog, and complete your testcase. Paste it into a script. You now have the following script:

testcase VerifyFuncTest ()TextEditor.Search.Replace.Pick ()Replace.Direction.GetItemCount ()Replace.Cancel.Click ()

Now use the Verify function to complete the verification statement. Change the lineReplace.Direction.GetItemCount () to Verify (Replace.Direction.GetItemCount (), 2)That is, the call to GetItemCount (which returns the number of radio buttons) becomes the first argument to Verify. The expected value, in this case, 2, becomes the second argument. Your completed script is:

testcase VerifyFuncTest ()TextEditor.Search.Replace.Pick ()Verify (Replace.Direction.GetItemCount (), 2)Replace.Cancel.Click ()

Verifying object attributes

Each kind of GUI object in an application has a variety of characteristics, called attributes. For example, a text field has the following attributes:

Caret position, which is the current position of the text insertion cursor, in (line, column) format. For example, a value of {1,1} means that the text insertion cursor is positioned on line 1, column 1.

Enabled, which is the current enabled status of the text field, either TRUE or FALSE.

Selected range, which is the beginning and ending position of the text string currently selected in the field, in (line, column) format. For example a value of {1,12,1,16} means that the selected text begins on line 1, column 12 and ends on line 1, column 16.


# 60 #


Selected Text, which is the string that is currently selected, if any, in the text field.

Text, which is the entire contents of the text field.

By recording verification statements for the values of one or more of an object’s attributes, you can determine whether the state of the application is correct or in error when you run your testcases. That is: did the feature you are testing have the expected result?

Note Attributes have been essentially rendered obsolete and have been replaced by properties.

Procedure To verify an attribute of a GUI object: Select Options/Recorder. Drive your application to the test state and press Ctrl+Alt. The Verify

Window dialog opens with the Attribute tab displayed.

The list box on the left shows the attributes for the current object. Select an attribute from the list box. The current value of the attribute

(that is, the value that exists when you are recording) is shown in the Attribute Value field.


# 61 #


If the current value of the attribute is not the value you want to test for at runtime, edit the attribute value field. The value specified in this field is the value you expect at runtime, that is, the baseline value.

Click OK to accept the attribute and its value. The Verify Window dialog is closed, and the Record Status window is opened.

The testcase will verify that the object has the attribute value selected. If not, an error is written to the results file.

Verifying all attributes By selecting the Verify All Attributes check box, you can record a test that verifies the state, contents, and value of a GUI object and any objects it contains. This is commonly called a smoke test or a Level 1 test. A smoke test uses the VerifyEverything method to verify every aspect of a particular GUI object.

Procedure To verify everything about a GUI object:

Select the Verify All Attributes check box. Click OK. The Verify Window dialog is closed, and the Record Status

window is opened.

Defining additional attributes If you need to, you can define and add your own attributes to the built-in hierarchy of GUI classes.

Recording the cleanup stage and pasting the recording

After performing the verification, continue to interact with your application. This is the cleanup stage. For example, in the sample testcase, cleanup means closing the document window without saving it.

When you have finished recording your testcase or just want to see what you have recorded, press Done on the Record Status window. The Record Testcase window is redisplayed. The Testcase Code field contains your interactions written as 4Test code.

Review the code and take the following actions:


# 62 #


Save the script file, as described in the next section.How the application state is recorded

If you chose DefaultBaseState as the application state, the testcase is recorded in the script file as:

testcase testcase_name ()

If you chose another application state, the testcase is recorded as:

testcase testcase_name () appstate appstate_name

If you chose “(none)” as your application state, the testcase is recorded as:

testcase testcase_name () appstate none

The recording statement


# 63 #


When you paste a recorded testcase (or other recorded actions, such as when you use Record Actions) into a script, SilkTest indents the code under a recording statement to facilitate playback.

Saving a script file

Procedure To save a script file, select File/Save. If it is a new file, you are prompted for its name and location. When saving a file, SilkTest does the following:

Saves a source file, giving it the .t extension; the source file is an ASCII text file, which you can edit

Saves an object file, giving it the .to extension; the object file is a binary file that is executable, but not readable by you

Example If you name a script file mytests and save it, you will end up with two files: the source file mytests.t, in the location you specify, and the object file mytests.to.

Procedure To save a new version of a script’s object file when the script fileis in view-only mode, select File/Save Object File.

RUNNING TESTS AND INTERPRETING RESULTS

Introduction You can run testcases individually, as a script, as a group of scripts (called a suite), or from a testplan if you are running QA Organizer. Whenever you run tests, SilkTest generates a results file, which indicates how many tests passed and how many failed, describes why tests failed, and provides summary information. You can invoke comparison tools from within the results file that pinpoint exactly how the runtime results differ from your known baselines. Testplan results files offer additional features, such as the ability to generate a Pass/Fail report or compare different runs of the testplan.

Running tests

You can run testcases individually, as a script, as a group of scripts (called a suite), or from a testplan if you are running QA Organizer.


# 64 #


Preparing to run tests If you plan to test Web applications, make sure you enable the correct browser extension(s), either during installation or using the Agent Options dialog. If you do not plan to test Web applications, you must disable all browser extensions using the Agent Options dialog. This is because the recovery system works differently when testing Web applications than when testing non-Web applications.

Procedure To enable browser extensions:

Select Options/Agent. Select the Extensions tab. Select the check box of each browser extension you want to enable.

Procedure To disable browser extensions:

Select the Options/Agent. Select the Extensions tab. Deselect the check boxes of all browser extensions.

Running a testcase Procedure To run a testcase:

Make sure that the testcase you want to run is in the active window. Select Run/Testcase. The Run Testcase dialog appears. The Testcase

listbox displays all the testcases contained in the current script.

Select a testcase, specify arguments (if necessary) in the Arguments field, separating multiple arguments with commas, and click Run. SilkTest runs the testcase and generates a results file.

Running testcases on platforms that do not support multiple tags


# 65 #


Multiple tags are supported by Windows 95 and Windows NT Agents. If you are running testcases using other Agents (such as on Motif), you can run scripts that use declarations with multiple tags by doing one of the following:

Checking the Disable Multiple Tag Feature check box in the Agent Options dialog (Compatibility tab).

Specifying the following in your script: Agent.SetOption (OPT_MULTIPLE_TAGS, FALSE)

After you have turned off multiple-tag support, 4Test discards all segments of a multiple tag except the first one.

Stopping a running testcase

Procedure To stop a running testcase before it completes:

If your te st application is on a target machine other than the host machine, select Run/Abort.

If your test application is running on your host machine, press Shift+Shift.

Grouping scripts in a suite

After you have created a number of script files, you might want to collect them into a test suite. A suite is a file that names any number of scripts. Instead of running each script individually, you run the suite, which executes in turn each of your scripts and all the testcases they contain. Suite files have a .s extension.

Procedure To create a suite:

Select File/New. The New dialog is displayed. Select the Suite radio button and click OK. An untitled suite file opens Enter the names of the script files in the order you want them

executed.

For example, the following suite file executes the find.t script first, the goto.t script second, and the open.t script third.

find.tgoto.t


# 66 #


open.t

Select File/Save to save the file.

Running a script or suite

Procedure To run the currently active script or suite:

Make sure the script or suite you want to run is in the active window. Select Run/Run. SilkTest runs all the testcases in the script or suite and

generates a results file.

Procedure To run a script or suite that is not currently open:

Select File/Run. Select the script or suite name from the Run dialog and browse the

directories to select the name of the script or suite file you want to run. Click OK. SilkTest runs all the testcases in the script or suite and

generates a results file.

Results filesWhen you run a testcase, script, suite, or testplan, SilkTest creates a

results file. The following figure shows a results file for a script (find.t). The menu bar includes the Results menu, which allows you to manipulate the results file and locate errors. The Results menu appears only when the active window displays a results file

Definition of results file


# 67 #


A results file provides information about the execution of the testcase, script, suite, or testplan. By default, the results file has the same name as the executed script, suite, or testplan, but with a .res extension (for example, find.res). To change the default name and directory of the results file, edit the Runtime Options dialog.

Overall summary By default, the results file displays an overall summary at the top of the file, including the name of the script, suite, or testplan; the machine the tests were run on; the number of tests run; the number of errors and warnings; actual errors; and timing information. The overall summary for find.t is shown in the preceding figure.

Procedure To hide the overall summary, click on the summary and select Results/Hide Summary. (You can change the default behavior by deselecting the Show Overall Summary check box in the Runtime Options dialog.

Individual test summary

You can see a summary for an individual test in a results file. For a script or suite results file, the individual test summaries contain timing information and errors or warnings. For a testplan results file, the individual test summaries contain the same information as in the overall summary plus the name of the testcase and script file.

Procedure To see an individual summary:

Click on a testcase line in a suite or script results file, or click on a test description in a testplan results file.

Select Results/Show Summary. The following figure shows the individual summary for a test in a testplan results file.


# 68 #


Hierarchical format of testplan results file

The format for the rest of a testplan results file follows the hierarchy of test descriptions that were present in the testplan. Test statements in the testplan that are preceded by a pound sign (# ) as well as comments (using the comment statement) are also printed in the results file, in context with the test descriptions.

Using the results file to find errors

You can expand the text of an error message yourself or have SilkTest find the error messages for you.Clicking on the red plus sign

Procedure To expand an error message to reveal the cause of an error, click on the red plus sign preceding the message. In addition, you see the call stack, which is the list of 4Test functions executing at the time the error occurred. In the following example of a results file, the error occurred in VerifySelText, which was called from Case_Back_Char on line 39 of find.t.


# 69 #


Using Find Error Procedure To find and expand the next error or warning message in the results file, select Edit/Find Error.

Tip To skip warning messages and find error messages only, deselect Find Error Stops at Warnings check box in the Runtime Options dialog. You can also use the Find, Find Next, and Go to Line commands on the Edit menu to navigate through a results file.

Navigating to errors in the script

SilkTest provides several ways to move from the results file to the actual error in the script:

Double-click in the margin next to an error line to go to the script file that contains the 4Test statement that failed.

Click on an error message and select Results/Goto Source. Click on an error message and press Enter.

Navigating to errors in the testplan


# 70 #


To navigate from a testplan test description in a results file to the actual test in the testplan, click on the test description and select Results/Goto Source.

What the box icon means

Some expanded error messages are preceded by a box icon and three asterisks, as shown in the following figure.

What happens when you click on the box icon depends on the error message.

If the error message relates to an application’s appearance, as in Bitmaps have different sizes, SilkTest opens the bitmap tool for your platform.

If the error message relates to an application’s behavior, as in Verify selected text failed, SilkTest opens the Difference Viewer. The bitmap tool compares baseline and results bitmaps, whereas the Difference Viewer compares actual and expected values for a given testcase.

Finding application appearance errorsWhen you click on a box icon followed by a bitmap-related error message,


# 71 #


SilkTest starts the bitmap tool, reads in the baseline and result bitmaps, andopens a Differences window and Zoom window.

Bitmap tool The following figure shows the bitmap tool. The baseline bitmap for the Font dialog in the Text Editor is compared to the result bitmap, which shows the Font dialog when the testcase was run. The Differences window shows the area of difference between the two bitmaps, that is, the different selection for Font Style.

The bitmap tool supports several comparison commands, which let you closely inspect the differences between the baseline and results bitmaps.

Finding application logic errors

To evaluate application logic errors, SilkTest provides the Difference Viewer, which you can invoke by clicking on the box icon following an error message relating to an application’s behavior.

Difference Viewer Clicking on the box icon invokes the Difference Viewer’s double-paned display-only window (shown in the following figure). It lists every expected (baseline) value in the left pane and the corresponding actual value in the right pane. SilkTest highlights all occurrences where expected and actual values differ. On color monitors, differences are marked


# 72 #


with red, blue, or green lines, which denote different types of differences, for example, deleted, changed, and added items. The highlight is not visible in the black-and-white

Fixing errors in a script Updating expected values

You might notice upon inspecting the Difference Viewer or an error message in a results file that the expected values are not correct. For example, when the caption of a dialog changes and you forget to update a script that verifies that caption, SilkTest will log errors when you run the testcase. To have your testcase run cleanly the next time, you can modify the expected values with the Update Expected Value command. Note that this command updates data within a testcase, not data passed in from the testplan. (For more information on testplan data, see Chapter 10, “Adding Data to a Testplan”.)

Procedure To fix incorrect values in a script: Make the results file active. Select Results/Update Expected Value. SilkTest replaces the expected

values in the script with the actual values found at runtime. Optionally, select Run/Testcase in order to run the test and confirm

that it now passes.


# 73 #


Using application states

When testing an application, you’re likely to have a number of testcases that have identical setup steps. For example, each testcase that exercises the sample Find dialog’s forward, case-sensitive searching ability has the same Setup stage.Example Each testcase needs to

Open a new document file Type text into the document Position the insertion point at the top of the file Select Find from the Search menu Select the forward (down) direction for the search Make the search case sensitive

Application states Rather than record the same steps over and over again, it is easier to record the steps as an application state and then associate the application state with the relevant testcases. An application state is the state you want your application to be in after the base state is restored but before you run one or more testcases. By creating an application state, you are creating reusable code that saves space and time. Furthermore, if you need to modify the Setup stage, you change it once, in the application state routine.

Application states canbe chained together

A testcase can have, at most, one application state associated with it. However, that application state may itself be based on another previously defined application state. For example, assume that

The testcase Find is associated with the application state Setup The application state Setup is based on the application state OpenFile The application state OpenFile is based on the built-in application

state, DefaultBaseState SilkTest would execute the programs in this order:

DefaultBaseState application state OpenFile application state Setup application state Find testcase

A base state is an application state


# 74 #


As this example illustrates, the built-in routine DefaultBaseState is itself an application state. The DefaultBaseState application state is responsible for restoring the application to the base state in the event the application fails or is corrupted during testcase execution or between testcases.

Base the final application state on DefaultBaseState

If a testcase is based on a single application state, that application state must itself be based on DefaultBaseState in order for the testcase to use the recovery system. Similarly, if a testcase is based on a chain of application states, the final link in the chain must be DefaultBaseState. In this way, SilkTest’s built-in recovery system is still able to restore the application to its base state when necessary.

Recording an application state

You define an application state before recording the testcase(s) associated with it. As with testcases, you can write an application state routine from scratch or you can use the Application State command on the Record menu.

Procedure To record an application state:

Open the file in which you want to place the application state. This can either be the test frame file for the application or the script file where the associated testcases are defined. If you put the application state in the test frame file, it will be available to all testcases. If you put it in the script file, it will be available only to testcases in that script file.

Open the application you want to test.

Select Record/Application State. The Record Application State dialog appears.


# 75 #


Type the name of your new application state in the Application State Name field.

Select an application state from the Based On drop-down list. Press the Start Recording pushbutton. SilkTest closes the Record

Application State dialog and displays the Record Status window. The Status field flashes Recording.

Drive your application to the state you want to record. At any point, you can record a verification by pressing Ctrl+Alt.

When you have finished recording an application state, press Done on the Record Status window. SilkTest redisplays the Record Application State dialog. The Application State Code field contains all the 4Test code you’ve just recorded. You can take the following actions:


# 76 #


Example Here is a sample application state that performs the setup for all forward case-sensitive searches in the Find dialog:

appstate Setup () basedon DefaultBaseStateTextEditor.File.New.Pick ()DocumentWindow.Document.TypeKeys ("Test Case<Home>")TextEditor.Search.Find.Pick ()Find.CaseSensitive.Check ()Find.Direction.Select ("Down")

Testing the application state

Before you run a testcase that is associated with an application state, you should make sure the application state compiles and runs without error.

Procedure To test an application state:

Make active the window that contains the application state and select Run/Application State. The Run Application State dialog appears, shown in the following figure.


# 77 #


Select the application state you want to run and click Run. If there are compilation errors, SilkTest displays an error window. Fix

the errors and rerun the application state.


# 78 #