Meter mode setting : Helps in optimizing the load generationby PATEL AKHILKUMAR on OCTOBER 3, 2011

Preface:Oftentimes, we have observed that the Jmeter samples of load scenarios don’t match with

previous runs or don’t match among different servers in standalone and client server

environment (distributed) etc. The results/samples depend on the many parameters like

SUT version, time of execution, data in back end, pacing of requests, think time between

requests and iterations and jmeter server/client performance etc. But these situations are

known and in general, load test engineers will try to setup and keep everything else

consistent during all the test execution. However, of if you still notice differences in

samples of hits per seconds, response time etc. during test execution from different

systems,  then you should first verify your load test setup. This blog will will help you to

identify the initial steps for handling situations where you notice such differences:

Reduce sampling impact on load generation:The load generation process can be impacted highly (or negligible) based on the usage of

the memory, cpu, disk and network by the jmeter server and/or client. In all of these

scenarios, the network traffic can impact load generation heavily if we are using

distributed load execution scenario and system is sending all the samples to client

immediately, depending on the network and network adapter traffic handling capacity as

well as on the client system performance (which is obvious in all of the cases). Even

jmeter sends the samples back to client as they are generated and client writes it to file

on receipt as  a default behavior means it uses network resources and client machine

memory , disk and cpu for same. So, if client machines network or other resources

performance are faulty, then it will impact on server’s load generation activity adversely.

To avoid this, set the sample sending mode appropriately and you will find more on mode

as moving forward on this post.

There are total 7 modes for sampling in JMeter:

1. Standard

2. Hold

3. Batch

4. Statistical

5. Stripped

6. StrippedBatch

7. Custom Implementation

1. Standard – Default mode

This will send the samples to client as they are generated and client will write it to

file on receipt in distributed environment. Same way it will try to write in file

immediately in standalone environment. This will use less memory of load generating

system but impact on load generation as the next requests will be sent after the

samples sending is completed by network layer. So if the receiving client’s network

is slow then the sample sending will be slow and which can impact on requests

sending to SUT. Still if the standard mode is required then try to reduce sampling

interval means reduce the client – server communication and sampling size. It is

advisable to run small test in standalone mode and server mode and if you found

difference then try to check the network and system performance.

2. Hold mode

In this mode, the sample data is held in an array in the memory and will be saved to

file at the end of run by client after sending this data to client in distributed mode

while it get saved at last in standalone environment (not advisable to use this mode in

standalone if disk IO is performing well). But as you must have already guessed, it is

a memory consuming mode and it can be a major cause for system crash . So, it is

advisable tthink thrice before using this mode. For small load test it is among best

load generation modes. Run small tests and watch the increase in memory usage to

understand how risky it is to use this mode and how best to use this mode.

3. Batch mode 

The system send samples in batch as per the defined threshold of samples or time. So

set following property appropriately:

          num_sample_threshold – number of samples in a batch (default 100)

          time_threshold – number of milliseconds to wait (default 60 seconds)

But sometimes it impact advertly if client network and system performance is not

good.

4. Statistical mode

This mode is mainly for summarized sampling and do not samples all the fields.

Also, the sampling rate is depends on the properties described with batch mode. The

samples would be grouped as per thread group name and sample label. It accumulates

only following fields and other fields that changes between samples will be ignored:

Fields which will be accumulated are: 1. Elapsed time, 2. Latency, 3. Bytes, 4.

Sample count and 5. Error count.

This mode somewhat reduces the samples data impact over the network and will use

very less resources of client as well in distributed environment. So it is advisable to

setup effiecient thresholds after consideration of client system performance, network

performance etc.

5. Stripped mode

This mode is nothing but standard mode without response data of successful samples.

6. StrippedBatch mode

This is nothing but the Batch mode without response data of successful samples.

7. Custom implementation mode

We can write our own custom sample sender class to customize the sampling

behavior and set the mode parameter to this custom sample sender class name.

This must implement the interface SampleSender and have a constructor which takes

a single parameter of type RemoteSampleListener.

For example:

public class PdfResponseSampleSender implements SampleSender,

Serializable { //The code will just show the basic

implementation and may be missing some required implementation

to fit in standard practice. private static final Logger log =

LoggingManager.getLoggerForClass(); //SampleSender h pave a

constructor which takes a single parameter of type

RemoteSampleListener. So following will be used for that.

private final RemoteSampleListener listener;

PdfResponseSampleSender(RemoteSampleListener listener)

{ this.listener = listener; } //It will be fired while sample

started and stopped public void sampleOccurred(SampleEvent

event) { //Basically it is advicable to create enum of mime

types. but here we just using it directoly in string variable

String pdfType="application/pdf"; SampleResult result =

event.getResult(); //Empty the response data if the media type

is not pdf. if(result.getMediaType().compateTo(pdfType)!=0)

{ result.setResponseData(new byte[0]); } try

{ listener.sampleOccurred(event); } catch (RemoteException e)

{ log.error("Error sending sample result over network ",e) } }

}

We can set mode=org.example.load.PdfResponseSampleSender into jmeter.properties and make sure that this class is reachable to jmeter.

Conclusion:I would say that we can resolve any load generation issue easily as jmeter is very flexible and supports customization for most of the activities of L & P testing.If you have any questions, please post those in the comments below, and I will try to answer those as soon as I can!PS: All opinions in this post are my personal and it may or may not be of Infostretch. Also, I have taken help of JMeter user manual and source code for this blog post.

Regular Expressions20.1 Overview

JMeter includes the pattern matching software Apache Jakarta ORO  

There is some documentation for this on the Jakarta web-site, for example a summary of the pattern matching characters

There is also documentation on an older incarnation of the product at OROMatcher User's guide   , which might prove useful.

The pattern matching is very similar to the pattern matching in Perl. A full installation of Perl will include plenty of documentation on regular expressions - look for perlrequick, perlretut, perlre, perlreref.

It is worth stressing the difference between "contains" and "matches", as used on the Response Assertion test element:

"contains" means that the regular expression matched at least some part of the target, so 'alphabet' "contains" 'ph.b.' because the regular expression matches the substring 'phabe'.

"matches" means that the regular expression matched the whole target. So 'alphabet' is "matched" by 'al.*t'.

In this case, it is equivalent to wrapping the regular expression in ^ and $, viz '^al.*t$'.

However, this is not always the case. For example, the regular expression 'alp|.lp.*' is "contained" in 'alphabet', but does not match 'alphabet'.

Why? Because when the pattern matcher finds the sequence 'alp' in 'alphabet', it stops trying any other combinations - and 'alp' is not the same as 'alphabet', as it does not include 'habet'.

Note: unlike Perl, there is no need to (i.e. do not) enclose the regular expression in //.

So how does one use the modifiers ismx etc if there is no trailing /? The solution is to use extended regular expressions , i.e. /abc/i becomes (?i)abc. See also Placement of modifiers   below.

20.2 Examples

Extract single string

Suppose you want to match the following portion of a web-page: 

name="file" value="readme.txt"> 

and you want to extract readme.txt . 

A suitable regular expression would be: 

name="file" value="(.+?)">

The special characters above are:

( and ) - these enclose the portion of the match string to be returned . - match any character + - one or more times ? - don't be greedy, i.e. stop when first match succeeds

Note: without the ?, the .+ would continue past the first "> until it found the last possible "> - which is probably not what was intended.

Note: although the above expression works, it's more efficient to use the following expression: 

name="file" value="([^"]+)"> where 

[^"] - means match anything except " 

In this case, the matching engine can stop looking as soon as it sees the first " , whereas in the previous case the engine has to check that it has found ">rather than say " > .

Extract multiple strings

Suppose you want to match the following portion of a web-page: 

name="file.name" value="readme.txt" and you want to extract both file.name and readme.txt . 

A suitable reqular expression would be: 

name="([^"]+)" value="([^"]+)" 

This would create 2 groups, which could be used in the JMeter Regular Expression Extractor template as $1$ and $2$.

The JMeter Regex Extractor saves the values of the groups in additional variables.

For example, assume:

Reference Name: MYREF Regex: name="(.+?)" value="(.+?)" Template: $1$$2$

Do not enclose the regular expression in / /

The following variables would be set:

MYREF: file.namereadme.txt MYREF_g0: name="file.name" value="readme.txt" MYREF_g1: file.name MYREF_g2: readme.txt

These variables can be referred to later on in the JMeter test plan, as ${MYREF}, ${MYREF_g1} etc

20.3 Line mode

The pattern matching behaves in various slightly different ways, depending on the setting of the multi-line and single-line modifiers. Note that the single-line and multi-line operators have nothing to do with each other; they can be specified independently.

Single-line mode

Single-line mode only affects how the '.' meta-character is interpreted.

Default behaviour is that '.' matches any character except newline. In single-line mode, '.' also matches newline.

Multi-line mode

Multi-line mode only affects how the meta-characters '^' and '$' are interpreted.

Default behaviour is that '^' and '$' only match at the very beginning and end of the string. When Multi-line mode is used, the '^' metacharacter matches at the beginning of every line, and the '$' metacharacter matches at the end of every line.

20.4 Meta characters

Regular expressions use certain characters as meta characters - these characters have a special meaning to the RE engine. Such characters must be escaped by preceeding them

with \ (backslash) in order to treat them as ordinary characters. Here is a list of the meta characters and their meaning (please check the ORO documentation if in doubt).

( ) - grouping [ ] - character classes { } - repetition * + ? - repetition . - wild-card character \ - escape character | - alternatives ^ $ - start and end of string or line

Please note that ORO does not support the \Q and \E meta-characters. [In other RE engines, these can be used to quote a portion of an RE so that the meta-characters stand for themselves.]

The following Perl5 extended regular expressions are supported by ORO.

(?#text)An embedded comment causing text to be ignored.

(?:regexp)Groups things like "()" but doesn't cause the group match to be saved.

(?=regexp)A zero-width positive lookahead assertion. For example, \w+(?=\s) matches a word followed by whitespace, without including whitespace in the MatchResult.

(?!regexp)A zero-width negative lookahead assertion. For example foo(?!bar) matches any occurrence of "foo" that isn't followed by "bar". Remember that this is a zero-width assertion, which means that a(?!b)d will match ad because a is followed by a character that is not b (the d) and a d follows the zero-width assertion.

(?imsx)One or more embedded pattern-match modifiers. i enables case insensitivity, m enables multiline treatment of the input, s enables single line treatment of the input, and x enables extended whitespace comments.

Note that (?<=regexp) - lookbehind - is not supported.

20.5 Placement of modifiers

Modifiers can be placed anywhere in the regex, and apply from that point onwards. [A bug in ORO means that they cannot be used at the very end of the regex. However they would have no effect there anyway.]

The single-line (?s) and multi-line (?m) modifiers are normally placed at the start of the regex.

The ignore-case modifier (?i) may be usefully applied to just part of a regex, for

example:

Match ExAct case or (?i)ArBiTrARY(?-i) case

Perl5 regular expressions

Here we summarize the syntax of Perl5.003 regular expressions, all of which is supported by the Perl5 classes in this package. However, for a definitive reference, you should consult the perlre   man page   that accompanies the Perl5 distribution and also the book Programming Perl, 2nd Edition from O'Reilly & Associates. We are working toward implementing the features added after Perl5.003 up to and including Perl 5.6. Please remember, we only guarantee support for Perl5.003 expressions in version 2.0.

Alternatives separated by | Quantified atoms

{n,m}Match at least n but not more than m times.{n,}Match at least n times.{n}Match exactly n times.*Match 0 or more times.+Match 1 or more times.?Match 0 or 1 times.

Atomso regular expression within parentheseso a . matches everything except \no a ^ is a null token matching the beginning of a string or line (i.e.,

the position right after a newline or right before the beginning of a string)

o a $ is a null token matching the end of a string or line (i.e., the position right before a newline or right after the end of a string)

o Character classes (e.g., [abcd]) and ranges (e.g. [a-z]) Special backslashed characters work within a character

class (except for backreferences and boundaries). \b is backspace inside a character class

o Special backslashed characters

\bnull token matching a word boundary (\w on one side and \W on the other)

\Bnull token matching a boundary that isn't a word boundary

\AMatch only at beginning of string

\ZMatch only at end of string (or before newline at the end)

\nnewline

\rcarriage return

\ttab

\fformfeed

\ddigit [0-9]

\Dnon-digit [^0-9]

\wword character [0-9a-z_A-Z]

\Wa non-word character [^0-9a-z_A-Z]

\sa whitespace character [ \t\n\r\f]

\Sa non-whitespace character [^ \t\n\r\f]

\xnnhexadecimal representation of character

\cDmatches the corresponding control character

\nn or \nnnoctal representation of character unless a backreference. a

\1, \2, \3, etc.match whatever the first, second, third, etc. parenthesized group matched. This is called a backreference. If there is no corresponding group, the number is interpreted as an octal representation of a character.

\0matches null character

Any other backslashed character matches itself Expressions within parentheses are matched as subpattern groups and

saved for use by certain methods.

By default, a quantified subpattern is greedy . In other words it matches as many times as possible without causing the rest of the pattern not to match. To change the quantifiers to match the minimum number of times possible, without causing the rest of the pattern not to match, you may use a "?" right after the quantifier.

*?Match 0 or more times

+?Match 1 or more times

??Match 0 or 1 time

{n}?Match exactly n times

{n,}?Match at least n times

{n,m}?Match at least n but not more than m times

Perl5 extended regular expressions are fully supported.

(?#text)An embedded comment causing text to be ignored.

(?:regexp)Groups things like "()" but doesn't cause the group match to be saved.

(?=regexp)A zero-width positive lookahead assertion. For example, \w+(?=\s) matches a word followed by whitespace, without including whitespace in the MatchResult.

(?!regexp)A zero-width negative lookahead assertion. For example foo(?!bar) matches any occurrence of "foo" that isn't followed by "bar". Remember

that this is a zero-width assertion, which means that a(?!b)d will match ad because a is followed by a character that is not b (the d) and a d follows the zero-width assertion.

(?imsx)One or more embedded pattern-match modifiers. i enables case insensitivity, m enables multiline treatment of the input, s enables single line treatment of the input, and x enables extended whitespace comments.

Ajax Testing with Jmeter Initially when I tried to do Load testing with Ajax I found that it was not an easy job to do, especially when DWR concept is involved, I didn't find any good solution on web, so decided to dig in and give it a try on my desk top and found that handling Ajax with DWR in Jmeter is pretty simple but not that easy.

Here I am assuming that Ajax is implemented using DWR.

Before start recording you need to consider which version of DWR is used, based on the version of DWR we need to choose the recording approach. This approach will vary with DWR 1.3 and above 1.3

So considering the steps involved for DWR version 1.3

1. Record the scripts as you do normally.2. Have the listener "View result tree" which will help in debugging the scripts.

Fig 1

From the above figure we can see that few of the DWR calls have been captured. One thing to be noticed is that right hand panel shows “Name/Value” pair and the method used is "POST". The value captured is in single line and initially default to understand what it is. NOTE: - Don’t edit the value from Jmeter UI, It will screw up the scripts. We need to edit it in JMX file, Use Text Pad or any other editor to edit the JMX file.

After editing notice that each DWR calls will have unique id, this ID will differ for each run. 

c0-id=323_1456789521

Handling this id in such a way that new value will be generated for each run. In order to do this generate random numbers, We can see that id is split in to 2 parts separated by "_" so for first 3 digit generate 1 random number and for last 10 digit generate another random number. After modification it will look some thing like this.

c0-id=${dwr1}_${dwr2} Make sure you edit in JMX file.

In Order to generate the Ramdom numbers use "Random Function". This function looks like this ${__Random(100,400,test)} for dwr1 variable${__Random(100000000000,200000000000,test1)} for dwr2 variable.

In order to access these value use an "User Defined Variable" you can see in the below figure

Fig 2

dwr1 and dwr2 is the variable name which should be used to generate the id value. Now save the Jmx file and restart the Jmeter. when u run the script 1st DWR call will run with out any problem (figure 1 you can see that there are 3 DWR calls recorded) but fallowing DWR calls may fail some times, If you face the problem for running the next DWR calls, You need to change the content type of the request to "text/plain". In order to do these add the "HTTP Header Manager" config element and set the content type to text/plain

Name field=Content-Typevalue=text/plain

Fig 3If you fallow above instruction you can easily handle the Ajax calls with DWR version 1.3

For DWR version greater than 1.3 needs to be handled in little bit different way

1. Record the scripts for the pages where Ajax is implemented.

2. Once recording is completed you should be able to see the request some thing like xyz/dwr/engine.js, if you don't find this request then close the browser and open it newly and start recording again. This request will only be recorded when new browser is opened.

3. This version of DWR uses two session variable

JSESSIONID and SCRIPTSESSIONIDYou need to capture these values and substitute it in JMX file where ever it is being used. In order to do this, add 2 RegularExpressionExtractor under dwr.engine.js request

For Jsessionid

Reference Name: jsessionidRegular expression: dwr.engine._origJSessionId = "(.*)";

You can get this jsessionid from cookie manager also.

For ScriptSessionid

Reference Name: scriptsessionidRegular expression: dwr.engine._origScriptSessionId = "(.*)";

Above things is w.r.t DWR 2.0 version.

4. Now use these variable to substitute the value and run the script.

Tip:-If you are not able to record any of the DWR calls then use HTTP Analyzer or any tool and create your own HTTP request and method should be POST.Posted by Pramod at 4:28 AM 

JMeter Ant Task We all know that the word Automation means, there should not be any human intervention for the task. Similarly w.r.t Jmeter we can achieve this by integrating with the ant build process. When ever a new code is checked in and build has happened then Jmeter should kick off immediately, run the scripts and produce the performance report, So now the question is how to do this?

Answer is simple,

1.To use the task, you must have JMeter installed. You must also include ant-jmeter-1.0.9.jar in your Ant classpath. Adding the jar to $ANT_HOME/lib will make this happen automatically.

2.Set the jmeterhome parameter to your JMeter install location, and the resultlog parameter to the name of a file to log the test results to.

You can either specify a single test plan using the testplan parameter, or multiple test plans using the testplans nested element. The testplans element is a standard Ant FileSet element.

This is an Ant task for automating running JMeter test plans. The task executes one or more JMeter test plans, and logs the results to a file.

Start by defining the task to make it available to your build script:

<taskdefname="jmeter"classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask"/>

Set the jmeterhome parameter to your JMeter install location, and the resultlog parameter to the name of a file to log the test results to.

You can either specify a single test plan using the testplan parameter, or multiple test plans using the testplans nested element. The testplans element is a standard Ant FileSet element.

<jmeterjmeterhome="c:\jakarta-jmeter-1.8.1"testplan="${basedir}/loadtests/JMeterLoadTest.jmx"resultlog="${basedir}/loadtests/JMeterResults.jtl"/><jmeterjmeterhome="c:\jakarta-jmeter-1.8.1"resultlog="${basedir}/loadtests/JMeterResults.jtl"><testplans dir="${basedir}/loadtests" includes="*.jmx"/></jmeter>

Optional JMeter arguments supported include specifying an alternate jmeter properties file (jmeterproperties), running remote servers specified in jmeter

properties file (runremote), and running the tests through a proxy or firewall (proxyhost, proxyport, proxyuser, proxypass).

Setting the failureProperty attribute will set the specified property to "true" in the event of a JMeter test failure. This gives you the opportunity to take further action such as send an email or fail the and build.

You can override JMeter properties (instead of modifying jmeter.properties) like this:

<jmeterjmeterhome="c:\jakarta-jmeter-1.8.1"testplan="${basedir}/loadtests/JMeterLoadTest.jmx"resultlog="${basedir}/loadtests/JMeterResults.jtl"><property name="request.threads" value="1"/><property name="request.loop" value="10"/></jmeter>

You may also specify additional JVM arguments to the JVM launched to run JMeter. Here is an example of how to specify JVM arguments:

<jmeterjmeterhome="c:\jakarta-jmeter-1.8.1"testplan="${basedir}/loadtests/JMeterLoadTest.jmx"resultlog="${basedir}/loadtests/JMeterResults.jtl"><jvmarg value="-Xincgc"/><jvmarg value="-Xmx128m"/><jvmarg value="-Dproperty=value"/></jmeter>

You can find XSLT file in "Jmeter\extras" folder jmeter-results-report.xsl, for generating a summary report from the result log file. The summary report is very similar to the default report created by the junitreport task. You can use the xslt task to create the report:

<xsltin="${basedir}/loadtests/JMeterResults.jtl"out="${basedir}/loadtests/JMeterResults.html"style="${basedir}/loadtests/jmeter-results-report.xsl"/>

Note: If you are using JMeter 2.1 or later, you must use the new xslt stylesheet(s) included in the JMeter extras directory. The new stylesheets have been modified to support the new JMeter log file format.

If you would like failure detail messages in the report output, you must configure JMeter to output that information to the result log. To do this, set the following property in your jmeter.properties file before running the test plans:

jmeter.save.saveservice.assertion_results=all

Note: As of JMeter 1.9RC2(?), the default results output format is now csv. It must be changed to xml in order to use the xslt task to create the html report:

jmeter.save.saveservice.output_format=xml

There is also another XSLT file that was contributed which generates an enhanced report that includes expandable details.

The report will look something like

this: 

ParametersAttribute Description

jmeterhome JMeter install location.

testplan The location of the test plan file.

resultlog The location of the result log file.

resultlogdir The directory to place result log files. When used, result log file names will match the test plan files names, with the extension renamed from .jmx to .jtl.

failureproperty The name of a property to set to "true" in the event of a test plan failure.

jmeterproperties The location of an alternate jmeter.properties file to use.

runremote If "true", runs remote servers specified in jmeter.properties. Default is "false".

proxyhost Host name of a proxy to run the tests through.

proxyport Port of the proxy host specified.

proxyuser Username of the proxy host specified.

proxypass Password of the proxy host specified.

Nested ElementsElement Description

testplans Use instead of testplan attribute when you want to specify multiple test plan files. This element is a standard Ant

property Use to specify additional JMeter properties (instead of modifying jmeter.properties file). Attributes include name and value.

jvmarg Use to specify JVM arguments to the JVM launched to run JMeter. The only attribute is value.

jmeterarg Use to specify additional JMeter command line arguments. The only attribute is value.Posted by Pramod at 10:48 PM 0 comments    Links to this post

Sunday, April 12, 2009

JDBC Testing with Jmeter Data base testing with Jmeter is simple. Here I am using Jmeter2.3 and Oracle 10g.

Some of the things needs to be taken care before start testing.

1. Place the JDBC JAR file in --> Jmeter\lib folder (For Oracle 10g it will be classes12.jar)

2.Need to know the JDBC Driver class, If you don't know where to find the Driver class information, Open the Jar file using winzip or any tool for that matter and edit the OracleDriver.class in text file and there you can find the JDBC Driver Class.( oracle.jdbc.driver.OracleDriver)

3. Create Test Plan and add thread group.

4. Add the Config Element-->”JDBC Connection Configuration” and fill in all the required information, It should look like below figure.

Fig 1

5. From the above figure notice that Variable name is highlighted, what ever name you give here will be shared with the "Jdbc request default". In my example I have given variable name as "SQL". Provide the JDBC Drive Class information which you got from OracleDriver.class file.

6. Add the Config element "JDBC Request"

Fig 2

7. Give the variable name as "SQL" this name should match with "JDBC Connection Configuration" variable name.

8. If you are executing select statement, Update statement chooses appropriate Query type and put your query there.

9. Add view result tree for debugging purpose and run the script.

Hope this information will help!!!!