TFS 2012 and Ranorex

TFS/VS/MTM 2012 and Ranorex

Page | 1

Team Foundation Server /

Visual Studio /

Testmanager 2012

and

Ranorex

Documentation

Sample Virtual Maschine

(VM: TFS2012_U2_Ranorex)

Martin Sedlmeier

S3CC GmbH

Mai 2013


Page | 2

Table of Contents

Overview ................................................................................................................. 3

Overview Infrastructure .......................................................................................... 5

Start the VM: ........................................................................................................... 6

Planning ................................................................................................................ 10

Implementation: .................................................................................................... 15

SCM & CI ............................................................................................................... 18

Integration (Part 1 and Part 2) ............................................................................... 22

Associate ............................................................................................................... 31

Troubleshooting .................................................................................................... 39


Page | 3

Overview This document describes a Virtual Machine (VM) which contains a sample

integration of Ranorex into the Application Lifecycle Management (ALM) - Process

with Team Foundation Server (TFS), Visual Studio (VS) and Test Manager (MTM).

The VM TFS2012_U2_Ranorex extends a VM provided by Brian Keller (Microsoft

Principal Technical Evangelist for ALM). Many thanks to Brian for this VM

(downloaded from http://blogs.msdn.com/b/briankel/).

Figure 1: Overview Integration

Planning:

The story starts with the Planning of the User Stories (PBIs) for a Sprint. Part of this

PBI is a tiny Test Case. After manual Testing via Testmanager this Test Case is

planned for UI-Automation, should be used in our Build Definition of our

Application (CI) and the (automated) Test Case should be part of some Regression

Tests (later on).

http://blogs.msdn.com/b/briankel/


Page | 4

The UI-Test (Automated with Ranorex) should start inside Visual Studio

(Developer), inside a Build Definition (e.g. Nightly Build) and inside Microsoft Test

Manager (Tester).

Implementation:

The manual Testrun for this Test Case could be finished successfully. Now, we are

able to automate this Test Case via Ranorex.

SCM & CI:

The automated Test runs pretty well in Ranorex, so it is time to add the (Ranorex-)

Solution to Source Control in TFS. Additionally we create a Build Definition for this

(Ranorex-) Solution and copy the result to a shared Drop-Folder.

Integration (Part 1):

As next step we integrate the UI-Test into the (Unit)-Testing Framework. This allows

us to start the automated UI-Test via TestExplorer in Visual Studio.

Integration (Part 2):

Now, we are able to integrate this automated UI-Test into the nightly build of our

Application.

Associate:

Finally we associate the automated UI-Test with the Test Case. We created a tiny

Lab Environment, so we are able to start the automated UI-Test from Testmanager.


Page | 5

Overview Infrastructure All Components are installed on a single VM. For sure, this shouldn’t be done in

real life, this VM is only for demonstrate reasons.

All Services are running under the Account Administrator. There is a Build- and Lab-

Environment configured (based on localhost).

All the work (e.g. automate UI-Tests, add to Source-Control) would be done under

the Account ./brian (Tribute on Brian Keller;-) ).

The VM is in a VMWARE-Format and runs quite well on a Win 8 Notebook. The VM

is configured with 4 GB Memory (should be enough) and 2 Processors. You need

VMWare-Player (free) for starting the VM locally.

All the Passwords (Accounts) are set to P2ssw0rd (Upper P – digit 2- lower ssw –

digit 0 (zero) – lower rd).

The software is provided with Trail-License Keys (10 days left). It is highly

recommended, that this keys will be replaced with valid MSDN-Licenses.

Please take a look to this document: http://download.microsoft.com/download/A/9/2/A9253B14-5F23-4BC8-9C7E-

F5199DB5F831/Working%20with%20the%20Visual%20Studio%202012%20Update%202%20RTM%20ALM%20Virtual%20Machine.docx

You will find some important information about the correct handling of the VM. It

describes the Hyper-V Version, but you will find some hints about time-settings in

the VM: The System Time in the VM is set to June 2012 (set by a tiny startup-task).

After using the VM, it is highly recommended, that you change the value in the File

C:\util\SetDateAndTime.bat to the current System Time in the VM (+ 5 min) before

you shutdown the VM.

The System Time will be set to this Value at the next startup of the VM.

http://download.microsoft.com/download/A/9/2/A9253B14-5F23-4BC8-9C7E-F5199DB5F831/Working%20with%20the%20Visual%20Studio%202012%20Update%202%20RTM%20ALM%20Virtual%20Machine.docx

http://download.microsoft.com/download/A/9/2/A9253B14-5F23-4BC8-9C7E-F5199DB5F831/Working%20with%20the%20Visual%20Studio%202012%20Update%202%20RTM%20ALM%20Virtual%20Machine.docx


Page | 6

Start the VM: There are some steps to do to bring the VM (and Services) up and running: Note: please leave the accounts always in this State (e.g. logged on).

Description Screenshot

Copy VM to a local Folder

(e.g. C:\_MyVMs)

Start the VM

(DoubleClick on VM-Config-File

C:\_MyVMs\TFS2012_U2_Ranore

x \TFS2012_Update2.vmx)

Note: you have to install VMWare

Workstation or VMWare Player

Because of the configured Lab-

Environment after starting the

VM the user Administrator is

automatically logged in. The Test

Agent is started and Online.

If the Status is set to

disconnected, please refer to the

section Troubleshooting at the

end of this document.

Start the Build Service:

via Team Foundation

Administrator Console Build

Configuration Start


Page | 7

The Build Service runs as a

“interactive process”: you have to

provide the Account-Password

(P2ssw0rd)

The Build-Service is starting:

Minimize the Windows

Important:

otherwise the UI-Tests will not

run

Close the TFS Admin Console:

File Exit


Page | 8

Leave the Account Administrator

as logged on and switch to user

Brian:

Menu VM send Ctrl+Alt+Del

switch User

Menu VM send Ctrl+Alt+Del

DoubleClick Account Brian

Keller

Type in Password (P2ssw0rd)

Open a RDP-Session to the

Account Administrator on

localhost: DoubleClick RDP-Icon

on the Desktop of the Account

Brian,

Type in the Password (P2ssw0rd)

for Account Administrator


Page | 9

Restore down the RDP-Window,

but do not minimize it.

Important:

otherwise the UI-Tests will not

run


Page | 10

Planning Back to our Story: in this section we will go to the Story Backlog, take a look to our

PBI and the defined Tasks and Test Cases. We will see the Definition of a Test Plan

for Release 1 and a query based Test Suite. We will run our Test Case manually and

we take a look to the Test Results of our Test runs. After all, we change the

Automation Status on our Test Case from not automated to planned.


Under the Account Brian:

Open Browser IE and go to

the TFS FF Web Access

view Backlog

In the Sprint Backlog of the

Sprint 3 you will find the

PBI “Customer can create a

new Service Ticket” with

some Tasks.

That means:

- the Implementation is

planned

- the manual Test is

planned

- the automate Test is

planned

- the integration is planned

.., and trackable in the

Daily scrum Meeting


Page | 11

The PBI contains a linked

Test Case.

In this State of the

Lifecycle of the PBI the

Automation State is set to

Not Automated.

After the Implementation

of the PBI (Task implement

UI) is done the manual

Tests could start

Open MTM (Testmanager)

Plan Contents

Release 1 Sprint 3

(query based)

You will find our Test Case.

Open Query behind this

Test Suite ( edit Query)


Page | 12

The Query behind the Test

Suite is defined with the

Iteration Path and the

State of the Test Cases.

Under Test you will find

our Test Case 3 times,

because we defined 3

configurations for it

(Browser).

Start a manual Testrun via

Run Run with options

Note: there is a Build

associate with the current

Testrun

Check Box Run all the tests

manually for a manual

Testrun

(we will come back for

automated Testrun)


Page | 13

Start the manual Test Run

(without Option Create

action recording) and save

the Result

( Save and Close)

You will find all Test Results

under View results

See all the Test Results

under Section Result

History.

DoubleClick on a specific

Test Result …


Page | 14

… will open a detail view.

After the manual Tests

could be finished

successfully the

Automation Status could

be changed to Planned …

… and the Status of the

Task Test the implemented

UI could be set to Done.


Page | 15

Implementation: In this Section we will automate the Testrun via Ranorex Test Suite.


Brian will find his Task

for automate the Test

Case.

Note:

The Task contains 3

Links (with LinkType):

Parent:

the PBI to this Task

Predecessor:

Task for manual Testing

(s. Planning)

Related:

the Test Case which

should be automated

The Test Steps in the

Test Case describes the

Test Scenario.


Page | 16

Open the Ranorex

Studio and go to the

Test Suite

FabrikamFiberUITesting

Note:

We run the Tests with 3

different Browsers, but

we use always the same

Ranorex-Recording

Modules.

(The single Setups for

the Browser are

disabled, because we

will start this Testrun

outside of the Ranorex-

Studio.

At this point we are

able to run the

Testautomation via

Ranorex Studio Run

Test Suite


Page | 17

This works fine…

… so Brian finished his

work quite well (and

could set the Status of

his Task to Done )


Page | 18

SCM & CI At this Point we have an automated Test Case (with a Ranorex-Solution). Now we

will add this Solution to the TFS-Source Control (SCM) and create a Build-Definition

for this Ranorex-Solution (CI).


Brian will find his Task for

adding the Ranorex

Solution to the TFS-Source

Control.

Open the Visual Studio (VS),

connect to the local TFS

VSALM and the Team

Project FabrikamFiber in

the Collection

FabrikamFiberCollection


Page | 19

Open the Source Control

Explorer and take a look to

the Ranorex-Solution.

You could work with this

Solution in Visual Studio …

… or in the Ranorex

Solution


Page | 20

In the Team Explorer under

the Section Builds you will

find the Build Definition

Ranorex Solution (Main)

The Build Definition is using

a customized Build

Template called DefaultTemplate11.1forRanore

x

This Template builds the

Ranorex Solution and …

… copies the output (=

Ranorex Binaries) to a

shared Drop Location


Page | 21

The first Part of the

Integration is done, Brian

leaves the Task-Status to In

Progress and set Remaining

Work to 3 (hours).


Page | 22

Integration (Part 1 and Part 2) The next Step in our scenario is the Integration of the Ranorex Solution into the

(Unit)-Testing Framework. This means, we would like to start the UI-Tests inside of

Visual Studio and not only via Ranorex Studio (Part 1).

After this Integration we will be able to add this UI-Test to our Build Definition

Fabrikam (Main) with UI-Testing of our Application FabrikamFiber (Part 2).


Brian is still working

on this Task.

Under the Solution FabrikamFiber.CallCenter

you will find a new

Test Project.


Page | 23

The Integration is

really simple: just

create a TestClass

with some Test

Methods (as always).

The Integration is

done via Ranorex

TestSuiteRunner.

Note:

This TestClass is using

the Ranorex-Binaries

located in the Drop

Location of the

Ranorex Build

Definition.

Open the Test

Explorer…


Page | 24

… and run the Tests

You will find the

Result under Test

Explorer Passed

Tests Output

report.rxzlog (Ranorex

– Log File)

Note:

the TestMethod *_All

starts the UI-Tests

with the Browser IE

and FireFox,

TestMethod

*_All_WithChrome

starts IE, FireFox and

Chrome


Page | 25

Now, we can add this

Tests to our new Build

Definition (in this

Case for the

Application-Solution)

Fabrikam (Main) with

UI.Testing

We are using the

Default Template.

Under Process

automated Tests 1.

Test Source we define

our TestClass (Test

Sources Spec) and

TestMethod (Test Case

Filter)


Page | 26

Queue a new Build …

Note:

Be sure, that the Build

Service is started, the

RDP-Session is open

and not minimized.

Queue a new Build …


Page | 27

As you can see in the

RDP-Window the UI-

Tests are running

under the Build

Service (Account

administrator)

You will find the Test

Results of this Build

under the Build-Log

(Team Explorer

Builds my Builds

DoubleClick on the

finished Build)

section test run

click on Test entry

opens the Test Result-

Window Right

Mouse Click View

Test Results Details


Page | 28

You will find the

Ranorex-Log in the

Section Collected Files

(on the end of the

Log).

If the test goes to the

Status Failed , you

could quite simple

create a Bug –

Workitem.


Page | 29

The Test Result

(including the

Ranorex- Log) will be

automatically

attached to the new

Bug.

Note:

Please set the

Iteration Path to the

current Release…

… and you will find

this Bug in your

Product Backlog (via

Web Access)


Page | 30

You could queue a

Build also in the Web

Access.

The Task is still not

finished, so Brian let

the Status on In

Progress and changed

the Remaining Work

to 1 (hour)


Page | 31

Associate Finally we have to combine our new TestMethod (implemented via Testing

Framework) and the original Test Case. This allows the Tester to run this automated

UI-Test within Microsoft Testmanager (MTM).


For associating a

TestMethod with a

Test Case, we have to

start Visual Studio

and open our tiny

Test Case.

There is a Query

available: (Team

Explorer Work

Items Queries

Shared Queries

Current Sprint

DoubleClick on

Query Test Case

Open the Test Case,

go to the Tab

Associated

Automation and take

a look to the set

Values.

Note:

The Automation

Status changed to

Automated


Page | 32

Open the

Testmanager and

you will find our Test

Case in the Test Plan

Release 1 under the

Test Suite Regression

Testing (Testing

Center Plan

Content)

Note:

This is a Query Based

Test Suite. Click on

Edit query…

… and take a look to

the Query behind

the Test Suite.


Page | 33

Run the Test (Testing

Center Test

Run Tests Release

1 Regression

Testing mark our

Test Case Run

with options …

We would like to run

this test

automatically (do

not check option

Run all the tests

manually leave the

Environment to

LabForRanorex and

start the Test run

Note:

The selected

Environment is on

the same VM, in Real

Life this should be on

different Maschines.


Page | 34

As you can see in the

RDP-Window the UI-

Tests will start under

the Account

Administrator.

After the Test Run is

finished you will find

the Test Result under

the Section

Attachments.

DoubleClick the

attached *.trx-File…


Page | 35

… will open the Test

Results inside Visual

Studio.

View Test Results

Details will open …

… a detail View of

the Test Results.

Under the section

Collected Files you

will find the Ranorex-

Log.


Page | 36

You could create a

bug directly under

the Section Tests

Create bug)

As you can see under

the Tabs Test Cases

and Links some

additional Infos

would be

automatically linked

to the Bug (open it

with DoubleClick).


Page | 37

You will find this bug

in the Product

Backlog.

You could find a

Summary about all

of the Test Runs in

Testing Center

Test Analyze Run.

Define the view

(Manual or

Automated run)


Page | 38

Our tiny Test Case is

still manual

runnable. Check the

option Run all the

Tests manually

(Testing Center

Test run Tests

Release 1

Regression Testing

Run with Options

Finally all the work is

done, so Brian could

change the State of

his Task to Done.


Page | 39

Troubleshooting

Testagent will not start

If the Test Agent does not start (State = disconnected), please go to the Test Agent

Configuration Tool (via Start search programs and files type in Test Agent)

and set the Password for the User of the Test Agent.

Ranorex will not start

If Ranorex will not start (Errormessage like : Ranorex License already in use…),

please restart the VM (but don’t forget to set the new time in

c:\util\SetDateAndTime.bat).

Documents

TFS 2012 and Ranorex