Less12 3 e_loadmodule_2

Copyright © 2008, Oracle. All rights reserved.

Scenario Basics

Introduction to Oracle Application Testing Suite: e-Load

Copyright © 2008, Oracle. All rights reserved.12 - 2

Introduction to e-Load

Load and scalability component


Integrated Authentication in e-Load

Integrated Authentication Module

• e-Load includes an authentication module for restricting access to the e-Load interface. Users can enable authentication through the e-Load administrator and then specify user accounts and permissions for e-Load.

• When authentication is enabled, users will then be forced to login before accessing e-Load. Customers who prefer to bypass the login process can disable e-Load authentication.


Key Concepts

What is a virtual user?

• A virtual user (VU) is an “user” created by e-Load.

• Several hundred can be created by each workstation.

• Each VU runs a script, or transaction, as recorded in e-Tester.

• Each VU can be customized to have specific attributes that vary think times, browser used, and so on.


Key Concepts

What can a virtual user do?

• Emulates an actual user.

• Goes through transactions just as a real user would.

• Handle dynamic pages and can use databanks.

• Virtual users function independently of each other.

• Virtual users run as threads inside a process spawned by the controller.


Key Concepts

What is a “load session?”

• A data collection interval from the start of a load test to the the end of it.

• Performance testing may be compromised of several load sessions.


Setting up a Test

Use the test plan and requirements to set the settings in e-Load

This will emulate a “realistic user” scenario


Workspaces

Open the Workspace where your scripts are stored


Profiles

There are three types of profiles

• Default– Scripts that have been recorded

• User Defined – Custom profiles that combine default profiles

• Scenario– A combination of default and user defined profiles used to

generate load


Using the Default Profiles

Scripts in the open workspace are the default profiles


User-Defined Virtual User Profiles

User-Defined Profiles allow QA Engineers to combine multiple scripts to act as one script


Creating a User Defined Profile

Select “Manage > User-Defined Profiles” option to create a custom profile

Click on “New”


User-Defined Profiles

There are 4 main sections in a User Defined Profile

• Prologue – play only once at the beginning. – Login script.

• Run – iterate over as many times as is specified in the Autopilot. – Transaction that you wish to load test.

• Epilogue – play only once at the end.– Logoff script.

• Errors – play only if an error occurs during the Scenario run. – Script that logs the user back in on error.


Adding Visual Scripts

Highlight a default profile script.

Click on the “Add” button to add the profile to the selected section.


Adding Synchronization Points

Sync points provide the ability to create realistic multi-user situations that may expose resource conflicts such as deadlocks

Multiple virtual users executing the script will reach a sync point at various times

• Depending on, for example, – Speed– Number of pages– Response times

Sync points cause each virtual user to wait until all virtual users have reached that sync point


Adding Synchronization Points

Add the Synchronization Point by clicking on “Add Sync.”


Creating Scenario Profiles

Select a profile from either the default profiles or user-defined profiles.

Click on the arrow to move the profile to the scenario profiles.

Multiple profiles can be added.

The same profile can be added multiple times.


Number of Virtual Users (per profile)


Delay Between Iterations


Virtual User Pacing


Virtual User Pacing

Virtual User Pacing - specifies the visual script playback delay for each virtual user. There are four options:

• Recorded - uses the originally recorded delay times from the e-Tester visual script. You can set minimum and maximum delay times (in seconds) that override the visual script delay times in the minimum and maximum edit boxes.

• Random - uses random times for virtual user pacing. You can set minimum and maximum delay times for random delay in the minimum and maximum edit boxes.


Virtual User Pacing

Recorded/Rnd - uses random delay times based upon the recorded user delay. e-Load a random number based on the actual user delay plus or minus the upper and lower percentage settings.

No Delay – requests the next page as soon as the current request has completed its download. Because this will request pages faster than a normal user could, it will become extremely difficult to make a correlation between the number of virtual users and actual users. This should only be used for volume testing where transaction speeds are the important factor, not virtual users


Workstation


Adding New Workstations

Select Manage > Systems…


Creating New Workstations

Click the “New” button

Add the name of the agent

Add the host name or IP



Groups allow for easy management of similar machines



Groups can be submitted instead of workstations

Example: 500 users must run in the exact same configuration. Given that there are two machines and 250 users can run from each machine, submitting the scenario using a group will allow the tester to send the same configuration to both machines in the group rather than having to add the same profile multiple times


Configure all Parameters


Browser Emulation

VUs can be set to emulate different types of browsers


Custom Browsers

Tools > Options > Custom Browsers


Connection Speed


Caching Type

No not cache – no information is stored in the browser cache between or during an iteration.

First Time User – VU’s Information is cached during an iteration but discarded when the next transaction begins.

Repeat User – VU’s utilize files cached during and after the first iteration


User Mode


Thick Client

Thick Client is nearly a full browser simulation

Has the ability to reference the DOM and Java Virtual Machine

Best used for sites with heavy client side Java applets

Very Resource Intensive

• Uses approximately 10MB/VU, but may use more depending on various factors


Thin Client

Thin Client is a very slimmed down version of a browser

It does not interact with the Java virtual machine, DOM or JavaScript.

It can still work with sites that use applets that do not require the Java virtual machine

The preferred method of generating load in most circumstances

Not Resource Intensive

• Uses approximately .4MB/VU, but may use more


Java Client

A highly scalable version of the Thin Client that executes a compiled e-Tester script.

This agent provides a flexible code interface for performing customized scripting operations.


IP Spoofing


Use Download Manager


Content Download Manager

Select the type of content you want e-Load Virtual Users to request from the server during a load test

• Flash objects

• ActiveX, Java CAB files

• and other content types


Content Download Manager

The Download Manager allows the QA Engineer to generate additional load on the network pipe:

With the Download Manager turned off

• More load is generated against the application. When objects are in the process of downloading, there is

essentially no load on the application. This is usually done in the beginning of an iterative load testing process

With the download manager turned on

• The engineer can simulate real user load and see how the application runs overall with all features. This is usually done toward the end of load testing.


Download Rules

Users can specify content that should be downloaded during load test


Download Rules

Users can specify content that should be downloaded during load test


Download Manager

Example: In order to generate load from .JS JavaScript files associated with a web page the Download Manager must be set to download anything from the <Script> tag.


Data Bank Control

To access Data Bank Control

• Submit Scenario

• Click on Configure Data Bank


Databank Control

Users can specify how e-Load should read and use databank records


VU Display Options


Virtual User Display

View VU playback as pages are accessed

View only selected VU pages

View only pages with errors


Run User Defined Test Cases


Generating Page Timers

Select Options > Preferences > Auto generate timers for all resources to include downloaded images is e-Reporter timers


Add to Autopilot

Click the “Add to Autopilot” button to submit the scenario into autopilot


Using the Autopilot

Use the Autopilot to set the virtual user ramp-up rate

• Determine a rate that allows the tester to:– Observe application performance at different user volumes– Focus on performance trends and characteristics– Let the application “settle” between virtual user increases

The Autopilot tab will load with the specified default selections and values specified in the e-Load options dialog.


e-Load Autopilot

Autopilot is used to automate control of the load test execution

• Specifies automatic start/stop times for the scenario

• Specifies characteristics of the rampup


Start and Stop Settings

Specifying when to start Virtual Users• Immediately• After Delay Of• At Specified Time

Specifying when to stop Virtual Users• When the Stop button is

pressed• After Iteration Count• After Delay Of• At Specified time


Virtual User Rampup

Add per step• Specify Percentage

– Good for large or odd numbers of VU’s

• Specify Number– Good for easy numbers of

VU’s

After every• Iteration count Of

– Helps get good data for a given number of VU’s

• Delay Of– Good if scripts take a long

time to execute an iteration


Start and Stop the Test

The autopilot can be controlled manually by using the green and red buttons.

The point of controlling Rampup automatically, manually, or both, is to allow e-Load to build accurate data for a given number of active VU’s


Start and Pause Autopilot

Use the “Start Autopilot” button to start adding VU’s

Use the “Pause Autopilot” to stop the rampup of VU’s


Session Start/Stop

Define how a session starts:

Save data for reporting: Specifies if virtual user data is saved for post testing analysis reports

Auto assign session name: When selected, the specified session name prefix, plus a four-digit increment number, is assigned when a new session starts

Session name prefix: Specifies a fixed name to add before the session name. Enter a name to use or “Default.” e-Load adds the increment number to the name you define. When set to “Default,” the name is “session” (for example, session0001).


Session Start/Stop

Define how a session ends:

Stop session on last VU completion: When selected, the session ends when the last virtual user has finished the run in the Autopilot.

Stop attached session after browser closes: When selected, all attached sessions automatically close when a browser closes

Terminate all agents at end of session: When selected, all e-Load Agents automatically close when a session ends


Session Start/Stop

Agent error handling:

Stop ramp-up on agent error: When selected, the autopilot stops submitting new virtual users if any virtual users fail to complete the initialization process

Drop failed agents from session: When selected, the autopilot stops submitting new users to the agent machine that had the failure if a virtual users fails to start or is set to orphaned for any reason


Session Start/Stop


Scenario Configuration

Once the scenario is complete, save it in order to re-run the scenario later by selecting the “Scenario > Save” menu option


Virtual User Grid

Overview of the Virtual Users Grid columns


Virtual User Grid

VU-Id - ID number for each virtual user.

Profile - Name of the user profile being run by the virtual user

Status - Current status for the virtual user.

• Running – user is transitioning to the next page

• Think – user is in think time delay before requesting the next page.

Iterations - Number of times the virtual user has played back the user profile

Failed – Number of failed iterations.


Virtual User Grid

Last Run Time - Average elapsed time to execute the scripts

Current Page - Title of the web page or application currently being accessed by the virtual user

Workstation – Name of the workstation that the VU is running on

Databank – Record that the VU is using

Current Error – Error encountered in the current iteration

Previous Error – Previous error encountered by the VU


Test Starts

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

00:0

0:00

00:0

2:53

00:0

5:46

00:0

8:38

00:1

1:31

00:1

4:24

00:1

7:17

00:2

0:10

Kb Rcvd/sec Virtual Users


Load Test Errors

Server Errors• Errors returned to the browser from web server• Includes 4xx and 5xx error codes – that is 404 Error – Not

Found

Windows Internet Interface (Wininet) Errors• Errors returned from WinInet• All 12xxx error codes – that is 12002 Error – Timeout

Page Content Errors• Failures returned by e-Load when validating content of

returned pages• Failed to find or failed to match

– Usually caused when incorrect page is returned by the server


Repositories

Allows you to access scripts in other locations

Tools > Options > Repositories


Session Start/Stop

Sessions specify the scope for e-Load data collection and reporting. The data collected while the load test is running is shown in the virtual user grid, runtime performance statistics. All load graphs, are saved to a database for post-testing analysis in the “Create Reports” tab.


Session Start/Stop

Sessions specify the scope for e-Load data collection and reporting. The data collected while the load test is running is shown in the virtual user grid, runtime performance statistics. All load graphs, are saved to a database for post-testing analysis in the “Create Reports” tab.


Reporting

Use these options to specify the parameters for refresh intervals and for creating profile timer names when generating timers for all resources.


Reporting

Intervals

Reporting data collection interval: Specifies how often the aggregate e-Load agent performance data is collected for reporting to the e-Load database.

User Interface refresh interval: Specifies how often to refresh the User Interface including the VU grid, ServerStats display, and sync point status.

Graph refresh interval: specifies how often to redisplay graphs in the “View Run Graphs” tab.


Reporting

VU Display:

Maximum total storage size: Specifies the maximum amount of data, in bytes, to send to the VU Display.


Reporting

Timer name format

Specifies the format of page timer names for reporting purposes when “Auto Generate Timers For All Pages” is selected. The following options are available:

Default: The default naming convention, that is, MyScript.Page[2]. This setting disables the other four options

URL resource name: The base file name, e.g. home.html is added

Frame name: The HTML frame name, that is, top, is added

Page number: The page number, that is, Page[2] is added

Profile name: The profile name, that is, MyScript is added


Scenario Defaults

The Scenario Defaults option consists of 7 sections

1. Main

2. Browser Settings

3. Extensibility

4. VU Display

5. Reporting

6. Error Handling

7. Advanced


Scenario Defaults


Scenario Defaults - Main

# VUs: Specifies the number of virtual users to run for the selected profile

System: Specifies the machine on which the virtual users will run

User Mode: The mode in which to run e-Load virtual users

Iteration Delay: Specifies the amount of time (in seconds) to wait between iterations of virtual user runs

VU Pacing(Think Time): Specifies the visual script playback delay for each virtual user

Use Databank: Specifies whether or not a databank will be used


Scenario Defaults - Browser Settings

Browser Emulation: Lists the available browsers.

Connection Speed Emulation: Lists the available connections speeds.

Cache Emulation: Lists the types of caching that can be used(1St Time User or Repeat User).

Use IP Spoofing: Specifies whether or not IP spoofing will be used.

Use Download Manager: All page objects that are defined in the Download Rules section will be downloaded.

Use WinInet: Specifies whether or not the Windows internet communication API will be used. If disabled (default), Java.net components are used.

Enable Cookies: cookies are used when set to “True.”


Scenario Defaults - Extensibility

Execute User Defined Tests: If set to True, text matching tests and server response test timers within the script will be executed.

Execute User Defined Scriplets: If set to “True,” custom scriplets within the script will be executed


Scenario Defaults - VU Display

View Error Responses: Displays errors encountered by the virtual user.

View All Responses: Displays all virtual user responses.

Show Request Headers: The request header information for web page resources requested by the selected virtual user appear in the virtual user display history list.

Show Response Headers: When selected the response header information for web page resources requested by the selected virtual user appear in the virtual user display history list.


Scenario Defaults - Reporting

Auto Generate Timers For All Pages: When set to true, timers will automatically be generated for all pages.

Auto Generate Timers for All Resources: Timers will automatically be generated for all resources.


Scenario Defaults - Error Handling

Object Download Errors Are Fatal: When true, a Web page object download error is considered a fatal error.

Zero Length Download Errors Are Fatal: A server response that indicates zero bytes length is considered a fatal error.

On Error Stop Virtual User: All virtual users are stopped if an error is encountered.

Request Timeout: Specifies the maximum amount of time a virtual user waits to access a page before timing out.


Scenario Defaults - Advanced

Maximum Users Per Process: Sets the maximum number of virtual users per single process. Once the number has been reached, a new process is started when additional users are added.

Maximum HTTP Threads Per Process: Specifies the number of threads per process for optimizing the pooling of virtual user requests into a finite number of I/O worker threads.

Maximum HTTP Connections Per User: Specifies the maximum number of server connections per process per server.


Exercise 2a

Using the current application, set up a load test based on the following site information then answer the questions on the following page:

• Expect to receive up to 10 simultaneous users at peak load

• Users are distributed in the following fashion: 50% obtain stock quotes, 30% view their portfolio, 20% execute a transaction (buy/sell)


Exercise 2b

Now add the following to the previous scenario:

• 50% of the users have IE 5 and 50% Firefox 2.0.

• All users access the internet via a LAN connection.

• Users’ browsers all make use of caching.

• It is important to know the individual page download times for each transaction.


Review 2

1. How many Scenario Profiles will you need assuming you can run all the users on the master machine?

2. What will you set for VU pacing and why?

3. What options will you set for “Browser Emulation”, “Connection Speed” and “Caching Type?”

4. Will you choose to download images? Why or why not?

5. How could you vary the search input for each VU executing the search script(s)?

6. How can you increase the load that you are generating against the application (list multiple ways)?


View Run Graphs and Statistics


Real-time Load Results

Virtual User performance/response times

• How increased load affects user experience.

Performance of application infrastructure under load

• How the underlying Web infrastructure affects performance to aid in identifying bottlenecks.


e-Load Results

Analyze real-time load test results


e-Load Results

Real-time Performance Statistics as load test is running.

Click “View Run Graphs” to see real time data.


Performance Statistics

Hits = Pages + Images + Frames

• If download images is unchecked, images are not counted

Page = all resources on one page

• If Download images is unchecked AND there are no frames, then Hits = Pages

Kilobytes = Kb transferred from server to clients

Transaction = Complete play of the visual script by a VU

Technology

Less12 3 e_loadmodule_2