Flex and Flash Automation Testing

2/27/11 3:06 PMFlex and Flash Automation Testing with PushToTest TestMaker

Page 1 of 18file:///Users/fcohen/svn/tm6/documentation/UserGuide/Chapters/TestMakerUserGuide-Flex.html

Flex and Flash Automation Testing

TestMaker Flex Automation provides developers with the ability to create Flex applications that usethe Adobe Flex Automation API. Use the TestMaker Object Designer (TOD) to create functionaltesting scripts for Flex applications, including regression testing, smoke testing, and integrationtesting. See the FlexFlashAMF_Tutorial tutorial for Flex load and performance testing.

Introduction

When working with the Test Object Designer for Flex automation, one should understand thefollowing terms:

Automation agent (or, simply, agent) - An agent facilitates communication between aFlex application and an automation tool. The Flex Automation Package includes a pluginthat acts as an agent between your Flex applications and the Designer testing tool.Designer uses AutoQuick custom recording and replaying agent for Flex automation.

Automation tool - Automation tools are applications that use the Flex application datathat is derived through the agent.

The following illustration shows the relationship between a Flex application, an agent, andan automation tool.

http://www.adobe.com/devnet/flex/samples/custom_automated.html



AutoQuick - is an agent that records and replays user interaction with a Flex application.This process is considerably more complex than simply recording user interaction for thepurpose of metrics logging.

ExternalInterface - is the External API, an application programming interface that enablesstraightforward communication between ActionScript and the Flash Player container– forexample, an HTML page with JavaScript. Adobe recommends using ExternalInterface forall JavaScript-ActionScript communication.

You can call an ActionScript function in Flash Player, using JavaScript in the HTML page.The ActionScript function can return a value, and JavaScript receives it immediately asthe return value of the call.

This functionality replaces the fscommand() method.

The ExternalInterface requires the user's web browser to support either ActiveX® or the NPRuntimeAPI that is exposed by some browsers for plug-in scripting. Even if a browser and operating systemcombination are not listed above, they should support the ExternalInterface class if they support theNPRuntime API.

Getting Started

This chapter aims at making the testers to know how to prepare a Flex application for testing. It alsoexplains various compiling options, launching the Flex application using TOD for recording andplayback of the designer scripts

Directory Structure

The testers should understand how to structure directories for automating Flex applications inTestMaker. In directory Editor/Resources/TOD/Flex/, you can see the directory structure as below



src - contains Flex application source code (.mxml)

includes - contains action script files used in the application (.as)

libs - contains automation library files such as AutoQuick.swc, automation_dmv.swc,automation_agent.swc, automation.swc, etc

extlib - contains any external library used in the application(.swc)

deploy - contains the compiled Flex application with automation API (.swf

Required Libraries

The required automation libraries:

AutoQuick.swc

automation_dmv.swc

automation_agent.swc

automation.swc

Compiling Flex Applications with Automation API

In order to achieve functional testing in Flex applications using TestMaker, the Flex applicationneeds to be compiled with the Flex automation API in the following ways.



Using command line arguments

The Flex application can also be compiled with an Automation API as a command line:

mxmlc -include-libraries+=../libs/automation.swc;../libs/automation_agent.swc;../libs/automation_dmv.swc;../libs/AutoQuick.swcMyApp.mxml

Using Adobe Flash Builder (Add SWC files to Flex Builder projects)

In Flex Builder, select your Flex project in the Navigator.

Select Project > Properties. The Properties dialog box appears.

Select Flex Compiler in the tree to the left. The Flex Compiler properties panel appears.

In the "Additional compiler arguments" field, enter the following command:

-include-libraries "libs\automation.swc" "libs\automation_agent.swc" "libs\automation_dmv.swc""libs\AutoQuick.swc"

In Flex Builder, the entries in the include-libraries compiler option are relative to the Flex Builderinstallation directory; the default location of this directory on Windows is C:\ProgramFiles\Adobe\Flex Builder 3.

If your application uses charts or the AdvancedDataGrid classes, you must also add theautomation_dmv.swc file.

Click OK to save your changes.

Editing Flex-config.xml

In order to have permanent compiling option, the Flex-config.xml located at C:\Program Files(x86)\Adobe\Adobe Flash Builder 4\sdks\3.5.0\frameworks should have the following entries:

<include-libraries>

<library>/libs/automation.swc</library>

<library>/libs/automation_agent.swc</library>

<library>/libs/automation_dmv.swc</library>

<library>/libs/AutoQuick.swc</library>

</include-libraries>

Using Apache Ant

The Adobe Flex Ant tasks provide a convenient way to build your Flex projects using an industry-standard build management tool. If you are already using Ant projects to build Flex applications,you can use the Flex Ant tasks to replace your exec or java commands that invoke the mxmlcand compc compilers. If you are not yet using Ant to build your Flex applications, you can takeadvantage of these custom tasks to quickly and easily set up complex build processes for yourFlex applications.



The Flex Ant tasks feature includes two compiler tasks, mxmlc and compc. You can use these tocompile Flex applications, modules, and component libraries. In addition, the Flex Ant tasksinclude the html-wrapper task that lets you generate custom HTML wrappers and the supportingfiles for those wrappers.

The build.properties and build.xml files are used for compiling the Flex applications with theautomation API. PushToTest recommends this technique to compile Flex applications. Thefollowing example shows an ant target:

build.properties:

FLEX_HOME=C:/Program Files/Adobe/Adobe Flash Builder 4/sdks/3.5.0

basedir = <TestMaker-installed-dir>/Editor/Resources/TOD/Flex;

SRC_DIR =${basedir}/src

LIBS_DIR =${basedir}/libs

DEPLOY_DIR = ${basedir}/deploy

build.xml

<project name="Flex Ant Tasks Build Script" default="compile Flex project">

<property file="build.properties" />

<taskdef resource="FlexTasks.tasks" classpath="${basedir}/libs/FlexTasks.jar"/>

<target name="init">

<delete dir="${DEPLOY_DIR}" />

<mkdir dir="${DEPLOY_DIR}" />

<copy file="${basedir}/AutoQuickEnv.xml" todir="${DEPLOY_DIR}/"/>

</target>

<target name="compile" depends="init">

<mxmlc file="${SRC_DIR}/Main.mxml" output="${DEPLOY_DIR}/Main.swf">

<load-config filename="${FLEX_HOME}/frameworks/Flex-config.xml"/>

<include-sources dir="${SRC_DIR}/includes" includes="builtin.as, Date.as, Error.as, Math.as,RegExp.as, XML.as"/>

<source-path path-element="${FLEX_HOME}/frameworks"/>

<compiler.debug>false</compiler.debug>

<compiler.include-libraries dir="${basedir}/libs" append="true">

<include name="automation.swc" />

<include name="automation_dmv.swc" />

<include name="automation_agent.swc" />

<include name="AutoQuick.swc" />

</compiler.include-libraries>

</mxmlc>



</target>

</project>

Security Configuration Settings

Browse tohttp://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager04.html,you will see the below flash application loaded on your browser.

For Linux, "/" is added as the trusted location.

For Windows, "C:\" is added as the trusted location.

These configuration settings are done for making the Designer communicate with the Flexapplication without any security restrictions.

Please refer this documentation to know more,

http://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager.html#117164

Recording user events on Application

In order to record all the user interactions with the Flex application,

Select Record --> choose browser as Flex Testing --> Enter target SWF(the target path shouldbe relative to the "<TestMaker-installed-directory>/Editor/Resources/TOD/Flex" directory) --> ok.

It launches the compiled Flex application in an Titanium webkit Window and the communicationbetween the automation API and the javascript on the flash container enables that the userinteractions has been written in to the designer test list. There is a delay of few seconds for startrecording after the Flex application is launched.

http://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager04.html%3C/a%3E

http://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager.html#117164



Playback of recorded scripts

In order to record all the user interactions with the Flex application, the recorded user interactionson the Flex application can be automated by playing back the recorded designer scripts.

Select Play --> choose browser as Flex Testing --> Run Test launches the Flex application and itperforms all the recorded user actions. The playback window will be closed once the designer isdone with performing all the recorded user interactions with the Flex application.



AutoQuick Overview

Using the AutoQuick example

The AutoQuick example is available as a separate download (autoquick_src.zip file) athttp://download.macromedia.com/pub/developer/autoquick_src.zip

Before you write a custom replaying agent, you should try the AutoQuick example, which showsyou what type of information an agent must record so that it can play back the user interaction.

The AutoQuick example shows you how to specify codecs in the XML file and create a map ofcodecs that you can use for value translation.

http://download.macromedia.com/pub/developer/autoquick_src.zip



1. Download and expand the autoquick_src.zip file:http://www.adobe.com/go/Flex_automation_agent_apps

2. Build the AutoQuick.swc file by using the instructions in Building the AutoQuick.swc file.

3. Copy the AutoQuick.swc file to a location that is accessible by your application's compiler.

4. Include the AutoQuick.swc library in your Flex application.

If you are using Flex Builder, add the AutoQuick.swc file by selecting Project > Properties > FlexBuild Path > Library Path. Make sure that the Merged Into Code option is enabled.

If you are using the command-line compiler, add an entry in your Flex-config.xml file that pointsto this SWC file; for example:

<include-libraries>

<library>c:/myfiles/Flex3/agent/simpler_replay/AutoQuick.swc</library>


Alternatively, on the command line, you can add the files by using the include-libraries option; forexample:

mxmlc -C:\home\dev\depot\Flex\sdk\bin>mxmlc -include-libraries+=c:/myfiles/Flex3/agent/simpler_replay/AutoQuick.swcc:/myfiles/Flex3/agent/myreplay/ReplayExample.mxml

Be sure to use the += operator to add the SWC file; this ensures that you do not replace otherSWC files in your library path, but rather append this SWC file to the path.

5. Create a sample application that allows some kind of user interaction. For example, you couldcreate a simple application that has just a Button control:

<?xml version="1.0" encoding="utf-8"?>

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml">

<mx:Script>

<![CDATA[

private function clickHandler():void {

b1.label = "clicked";

}

]]>

</mx:Script>

<mx:Button id="b1" label="click me" click="clickHandler()"/>



</mx:Application>

6. Flex Builder creates an HTML wrapper for you. If you use the command-line compiler, youmust create one manually. You can use the following sample wrapper:

<html >

<head>

<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>

<title>ReplayExample</title>

</head>

<body>

<object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000"

id="ReplayExample" width="100%" height="100%"

codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab">

<param name="movie" value="ReplayExample.swf" />

<param name="quality" value="high" />

<param name="allowScriptAccess" value="sameDomain" />

<embed src="ReplayExample.swf" quality="high"

width="100%" height="100%" name="applicationReplayExample"

quality="high"

allowScriptAccess="sameDomain"

type="application/x-shockwave-flash"

pluginspage="http://www.macromedia.com/go/getflashplayer">

</embed>

</object>

</body>

</html>

7. Copy the wrapper and the application SWF file to your web server. You must deploy your



sample application to a web server.

8. Copy the AutoQuickEnv.xml file from the autoquick_src.zip file to the same directory on yourweb server as your sample application SWF and wrapper files.

Now that you have created an application that uses the AutoQuick toolbar, you can record andplay back events with it.

Record with the AutoQuick toolbar

1. Request the application SWF file. For example:

http://localhost:8100/replay/default.htm

2. The application opens with a Button control and the Automation pop-up window.

3. Click the Record button in the Automation pop-up window.

4. Click the button that you created in your application. The button's label should change from"click me" to "clicked".

5. Click the Stop button in the Automation pop-up window. The text area of the Automation pop-up window now contains the recorded interaction with the Flex application.

6. Copy the text from the Automation pop-up window to a text editor. It should look like thefollowing:

<Records>

<Step

id="automationClassName{FlexApplication

string}automationName{ReplayExample string}label{

string}automationIndex{index:-1 string}id{ReplayExample string}

className{ReplayExample string}|automationClassName{FlexButton

string}automationName{click%20me string}label{click%20me

string}automationIndex{index:0 string}id{b1 string}className

{mx.controls.Button string}"

method="Click">

<Args value=""/>

</Step>

</Records>



The <Records> tag wraps the entire script. Each event in the script is represented by a <Step>tag. Within the step tag, <Args> child tags can describe arguments to the event. For example, ifthe event is a MouseClick event, the <Args> child tags define what keys were pressed at the timeof the event.

The id attribute of the <Step> tag contains the entire control hierarchy for each target of an event.Each object in the id attribute is separated by a pipe (|), with the top-most object in the hierarchyoccurring first. For example, in an application that has a TextInput control inside an HBoxcontainer, you would have an id attribute similar to the following:

Application | HBox | TextInput

Each object in the hierarchy is defined by a set of properties that use the following syntax:

property_name{value type}

The ReplayExample sample shows the following information about the application:

automationClassName{FlexApplication string}

automationName{ReplayExample string}

label{string}

automationIndex{index:-1 string}

id{ReplayExample string}

className{ReplayExample string}

It also shows the following information about the Button control, including the label:

automationClassName{FlexButton string}

automationName{click%20me string}

label{click%20me string}

automationIndex{index:0 string}

id{b1 string}

className{mx.controls.Button string}

The object definitions contain values for all of the attributes that are defined in theAutoQuickEnv.xml file for objects of that type. This ensures that the object can be uniquelyidentified in the script. This entry in the id attribute identifies the Button control as an object withthe automationClassName property set to FlexButton, automationName set to label, label is "clickme", automationIndex is 0, id is b1, and the canonical classname is mx.controls.Button.

The method attribute of the <Step> tag defines the event that was recorded.

Play back the recorded session with the AutoQuick toolbar

Restart your Flex application by either refreshing the browser or closing and reopening thebrowser.



Copy the recorded script from your text editor and paste it into the Automation pop-up window'stext area.

Click the Play button. The Button control's label should change from "click me" to "clicked."

About the AutoQuick Example

The AutoQuick example uses a variety of assets:

AQToolBar.mxml -- Defines the floating window that provides record, stop, and play functions.This component calls the methods of the helper classes.

AQAdapter.as -- Defines the agent class for the AutoQuick example. This class is a mixin, whichtriggers a call to its static init() method from the SystemManager class when the application startsup.

AutoQuickEnv.xml -- Defines what components and their methods, properties, and events canbe recorded with the automation API. This is similar to the AutomationGenericEnv.xml file that isused by the CustomAgent example.

codec.* class files -- The classes in the codec package are used to convert ActionScript types toagent-specific types; you can think of these classes as providing serialization for the ActionScripttypes. For example, the AutoQuick example uses the DatePropertyDecode class to convert Dateobjects to an encoded form. The serialized object can then be loaded by the agent at run time.

Customizing the AutoQuick example

Two classes that you will probably customize are the AQToolBar.mxml file and the AQAdapter.asfile.

Adding functionality to the AQToolBar.mxml file is just like editing any other custom Flexcomponent. You can rebrand it, resize it, or change any of its properties to make it work withinyour application framework. For example, you could add an HTTPService call to a PHP page thatlogs the results of the recording. This would let you create a web-based, reusable eventrecording tool, similar to the CustomAgent example.

You can also customize the AQAdapter class; for example, you could add information to what isrecorded. The <Step> and <Args> data is defined in the recordHandler() method of this class.You can edit this class to add properties. For example, the <Args> values are currentlyrepresented as numeric values. If the user holds the Shift key down while clicking a Buttoncontrol, the agent records <Args value="2"/>. You could add logic to that class that includeshuman-readable values, such as <Args value="ShiftKey"/>.

After you have customized the AutoQuick example's source files, you must rebuild the SWC file.For more information, see Building the AutoQuick.swc file.

Building the AutoQuick.swc file

The autoquick_src.zip file contains all of the classes that you need to build the AutoQuick.swcfile. When you first unzip the file, or if you make changes to the source files, you will need to buildyour own SWC file.



To build your own AutoQuick.swc file from the AutoQuick source files, compile the custom SWCfile by using the following compc command:

C:\Flex\sdk\bin>compc

-source-path+=c:/myfiles/Flex3/agent/replay/AutoQuick

-output c:/myfiles/Flex3/agent/replay/AutoQuick.swc

-include-classes AQAdapter custom.CustomAutomationClass AQEnvironment

custom.CustomAutomationEventDescriptor custom.CustomAutomationMethodDescriptor

custom.CustomAutomationPropertyDescriptor custom.utilities.EnvXMLParser

IAQCodecHelper IAQMethodDescriptor IAQPropertyDescriptor codec.ArrayPropertyCodec

codec.AssetPropertyCodec codec.AutomationObjectPropertyCodec

codec.ColorPropertyCodec codec.DatePropertyCodec codec.DateRangePropertyCodec

codec.DateScrollDetailPropertyCodec codec.DefaultPropertyCodec codec.HitDataCodec

codec.KeyCodePropertyCodec codec.KeyModifierPropertyCodec codec.ListDataObjectCodec

codec.ScrollDetailPropertyCodec codec.ScrollDirectionPropertyCodec

codec.TabObjectCodec codec.TriggerEventPropertyCodec

The exact file paths may be different on your computer.

Working Mechanism

Architecture diagram of TestMaker's Flex automation:



Adobe Automation Libraries

Compiling Applications for Testing

If you do not use run-time testing, you must compile applications that you plan to test with thetesting libraries. The functional testing classes are embedded in the application at compile time,and the application typically has no external dependencies for automated testing at run time.

For information about run-time testing, see Use run-time loading.

When you embed functional testing classes in your application SWF file at compile time, youincrease the size of the SWF file. If the size of the SWF file is not important, you can use thesame SWF file for functional testing and deployment. If the size of the SWF file is important, youtypically generate two SWF files: one with functional testing classes embedded and one without.

To compile the Flex application for testing, you must reference the automation.swc andautomation_agent.swc files in your include-libraries compiler option. If your application uses



charts or the AdvancedDataGrid classes, you must also add the automation_dmv.swc file. Youmight also be required to add automation tool-specific SWC files; If you are using the AutoQuickexample, you must add the AutoQuick.swc file.

By default, Flex Builder includes the automation SWC files in its library path, but you must addthese libraries by using the include-libraries compiler option.

When you create the final release version of your Flex application, you recompile the applicationwithout the references to these SWC files. For more information about using the automation SWCfiles, see the Flex Automation Release Notes.

If you do not deploy your application to a server, but instead request it by using the file protocolor run it from within Adobe Flex Builder, you must put the SWF file into the local-trusted sandbox.This requires configuration information that is separate from the SWF file and the wrapper.

To include the SWC files, you can add them to the compiler's configuration file or as a command-line option. For the SDK, the configuration file is located at sdk_install_dir/frameworks/Flex-config.xml. To add the automation.swc and automation_agent.swc libraries, add the followinglines to the configuration file:

<include-libraries>

...

<library>/libs/automation.swc</library>

<library>/libs/automation_agent.swc</library>


You must uncomment the include-libraries code block. By default it is commented out in theconfiguration file.

You can also specify the location of the SWC files when you use the command-line compiler withthe include-libraries compiler option. The following example adds automation.swc andautomation_agent.swc files to the application:

mxmlc -include-libraries+=../frameworks/libs/automation.swc;

../frameworks/libs/automation_agent.swc MyApp.mxml

Explicitly setting the include-libraries option on the command line overwrites, rather thanappends, any existing libraries that you include in the configuration file. As a result, if you add theautomation.swc and automation_agent.swc files by using the include-libraries option on thecommand line, ensure that you use the += operator. This appends rather than overwrites theexisting libraries that are included.

To add automated testing support to a Flex Builder project, you also add the SWC files to theinclude-libraries compiler option.

Add SWC files to Flex Builder projects

In Flex Builder, select your Flex project in the Navigator.

i) Select Project > Properties. The Properties dialog box appears.

ii) Select Flex Compiler in the tree to the left. The Flex Compiler properties panel appears.

iii) In the "Additional compiler arguments" field, enter the following command:



-include-libraries "sdks\3.0.0\frameworks\libs\automation.swc""sdks\3.0.0\frameworks\libs\automation_agent.swc"

In Flex Builder, the entries in the include-libraries compiler option are relative to the Flex Builderinstallation directory; the default location of this directory on Windows is C:\ProgramFiles\Adobe\Flex Builder 3.

iv) If your application uses charts or the AdvancedDataGrid classes, you must also add theautomation_dmv.swc file.

v) Click OK to save your changes.

Limitations

TestMaker Flex automation has few limitations and we would like to rectify all these issues assoon as possible. We appreciate great help from the Flex testing community for helping us infixing these issues, including the following:

It works only for Flex applications compiled on SDK 3.5.0 and we are looking forward tomake it working for SDK 4.1.0

Few of the user actions like tab change and few user interactions on the customcomponents have issues when playing back the Designer Scripts.

For some of the actions, the waits between the designer steps on executing the actions isnot proper.

Some of the bugs in AutoQuick needs to be solved to increase the stability of the Flex automationusing TestMaker.

Example

A sample application is also shipped with TestMaker 6 in order to make the tester to understandthe ease of the test automation on Flex applications.

Record --> choose browser as Flex Testing --> Enter target SWF as "Flexstore.swf" --> ok

Flexstore.swf is pre-compiled with all necessary automation API files and its ready for testers todo recording and playback of the designer scripts for Flex automation testing.

Resources

Portions of this chapter include content from Flex Automation API documentation.

http://www.adobe.com/devnet/Flex/samples/custom_automated.html

http://download.macromedia.com/pub/documentation/en/Flex/2/at_api.pdf

http://livedocs.adobe.com/Flex/3/html/help.html?content=agents_7.html

http://www.adobe.com/devnet/flash/articles/external_interface.html

http://www.mozilla.org/projects/plugins/npruntime.html

http://jcheng.googlecode.com/files/Automation2008.pdf

http://www.adobe.com/devnet/flex/samples/custom_automated.html

http://download.macromedia.com/pub/documentation/en/flex/2/at_api.pdf

http://livedocs.adobe.com/flex/3/html/help.html?content=agents_7.html

http://www.adobe.com/devnet/flash/articles/external_interface.html

http://www.mozilla.org/projects/plugins/npruntime.html

http://jcheng.googlecode.com/files/Automation2008.pdf



Additional documentation, product downloads and updates areat www.PushToTest.com. While the PushToTest TestMaker software isdistributed under an open-source license, the documentation remains (c)2011 PushToTest. All rights reserved. PushToTest is a trademark of thePushToTest company.

http://www.pushtotest.com/

http://www.pushtotest.com/

Documents

Flex and Flash Automation Testing