Performance Testing With Open Source Tool JMeter

7/30/2019 Performance Testing With Open Source Tool JMeter

1/72

Author :Vikas Singh


2/72

What is Performance Testing??? Performance testing is defined as the technical

investigation done to determine or validate the speed,scalability, and/or stability characteristics of theproduct under test. Performance-related activities,such as testing and tuning, are concerned withachieving response times, throughput, and resource-utilization levels that meet the performance objectivesfor the application under test.


3/72

Key Terms of Performance Testing Performance test :To determine or validate speed, scalability, and/or stability.

Load test :To verify application behavior under normal and peak load conditions.

Stress test : To determine or validate an applications behavior when it is pushed beyond normal orpeak load conditions.

Baselines :Creating a baseline is the process of running a set of tests to capture performance metricdata for the purpose of evaluating the effectiveness of subsequent performance-improving changes tothe system or application.

Benchmarking :Benchmarking is the process of comparing your systems performance against abaseline that you have created internally or against an industry standard endorsed by some otherorganization.


4/72

Why Performance Test? Speed - Does the application respond quickly enough

for the intended users?

ScalabilityWill the application handle the expecteduser load and beyond?

StabilityIs the application stable under expectedand unexpected user loads?

Confidence Are you sure that users will have apositive experience on go-live day?


5/72

SpeedUser Expectations

Experience

Psychology Usage

System Constraints

Hardware

Network

Software


6/72

ScalabilityHow many users before it gets slow?

before it stops working?

will it sustain? do I expect today?

do I expect before the next upgrade?

How much data can it hold? Database capacity

File Server capacity

Back-up Server capacity

Data growth rates


7/72

StabilityWhat happens if

there are more users than we expect?

all the users do the same thing? a user gets disconnected?

there is a Denial of Service Attack?

the web server goes down?

we get too many orders for the same thing?


8/72

Confidence If you know what the performance is

you can assess risk.

you can make informed decisions. you can plan for the future.

you can sleep the night before go-live day.

The peace of mind that it will work on go-live dayalone justifies the cost of performance testing.


9/72

Core Performance-Testing Activities The seven core performance-testing activities can be summarized as follows:


10/72

Performance Testing ToolsOpen Source Tools

JMeter

http_load The Grinder

SiegeCommercial Tools

LoadRunner (around $200,000 to own!)WebLoad

Third-party $ervices


11/72

Why We Chose JMeter for Internal

TestingEasy to install and use

Free! Java most platforms Just download and run!

Feature-rich Post forms Record from browser Load test data from files Add logic, variables & functions Run one test from multiple machines Test many protocols, not just HTTP e.g .Http , Https , Ftp , Smtp


12/72

JMeter's Nomenclature

User -> Thread

Request -> Sampler

Report -> Listener


13/72

Test Plan The Test Plan is where the overall settings for a test are specified. Static variables can be defined for values that are repeated throughout a test, such as server

names.And can be accessed by the $(server) variable inside the test plan. Selecting Functional Testing instructs JMeter to save the additional sample information -

Response Data and Sampler Data - to all result files. This increases the resources needed torun a test, and may adversely impact JMeter performance.


14/72

Thread GroupA Thread Group defines a pool of users that will execute a particular test case against yourserver. In the Thread Group GUI, you can control the number of users simulated (num ofthreads), the ramp up time (how long it takes to start all the threads), the number of

times to perform the test, and optionally, a start and stop time for the test.


15/72

Configuration Element


16/72

Sampler

CSV Data Set Config: CSV Data Set Config is used to read lines from a file, and split theminto variables. By default, the file is only opened once, and each thread will use a differentline from the file. Lines are read at the start of each test iteration. The file name andmode are resolved in the first iteration.


17/72

HTTP Cookie Manager:

It stores and sends cookies just like a web browser. If there is an HTTP Request and theresponse contains a cookie, the Cookie Manager automatically stores that cookie and willuse it for all future requests to that particular web site. Each JMeter thread has its own"cookie storage area". So, if you are testing a web site that uses a cookie for storing sessioninformation, each JMeter thread will have its own session. Received Cookies are stored as

JMeter thread variable.Cookie stored in the Cookie Manager: This gives you the opportunity to use hardcodedcookies that will be used by all threads during the test execution. Not Recommended.


18/72

HTTP Request Default


19/72

Result of HTTP Request Default


20/72

User Defined Variable


21/72

Result of UDV


22/72

FTP Request Default


23/72

JDBC Connection Configuration


24/72

Controllers


25/72

Simple ControllerThe Simple Logic Controller lets you organize your Samplers and other Logic Controllers.

Unlike other Logic Controllers, this controller provides no functionality beyond that of a

storage device


26/72

Simple Controller result


27/72

Loop ControllerLoop Controller in JMeter will loop through the request

defined under it certain number of times, in additionto the loop value specified for the Thread Group.


28/72

Results of Loop Count ControllerIt will iterate through the samplers as given in Loop

Count.


29/72

Once only ControllerThe Once Only Logic Controller tells JMeter to process

the controller(s) inside it only once,and pass over anyrequests under it during further iterations through the

test plan. If it is defined under loop controller havingLoop Count = 3, it will execute the sampler under onceonly controller once.


30/72

Once Only Controller Results of Once only Controller: Loop Count under

Loop controller is defined to be 3. But once onlycontroller executes sampler under it once. But Once

only controller have no effect on Loop count definedunder thread Group.


31/72

The include controller is designed to use an external jmx file.

To use it, add samples to a simple controller, then save the simple

controller as a jmx file.

Logic Controller

Include Controller


32/72

Logic Controller

Include Controller contd


33/72

Runtime ControllerThe Runtime Controller controls how long its children

are allowed to run.


34/72

If ControllerThe If Controller allows the user to control whether the

test elements below it (its children) are run or not.

Condition: Conditions are given to make sure that all thesamplers below if controller are executed only whencondition is met.

Evaluate for all children: Evaluate for all children, is

given to evaluate condition for all children. Otherwiseit is evaluated at the entry point


35/72

Results of thread groupThread group = 2


36/72

ResultsResults: Note that as per condition sampler1 is executed

for both the thread group but, samplers under ifcontroller are executed only for thread 2.


37/72

Module ControllerThe Module Controller provides a mechanism forsubstituting test plan fragments into the current testplan at run-time.

A test plan fragment consists of a Controller and allthe test elements (samplers etc) contained in it. Thefragment can be located in any Thread Group, or onthe WorkBench . If the fragment is located in a Thread

Group, then its Controller can be disabled to preventthe fragment being run except by the ModuleController. Or you can store the fragments in a dummyThread Group, and disable the entire Thread Group.


38/72

Module Controller contd..Module to Run: The module controller provides a list of

all controllers loaded into the gui. Select the one youwant to substitute in at runtime.


39/72

As the other controllers are not made deactive so allcontrollers have been executed once. Under Modulecontroller Simple Controller have been placed. Simple

Controller have been executed twice.


40/72

Timers


41/72

Constant TimersTo pause each thread for the same amount of timeConstant timer is used.

Timer can be added as a child to thread group or as achild to sampler. If added to thread group it will delaythe entire thread group for the time specified. And if itis added to sampler then it will delay the sampler bythe given amount of time.


42/72

Uniform Random TimerThis timer pauses each thread request for a randomamount of time, with each time interval having thesame probability of occurring. The total delay is the

sum of the random value and the offset value.Total Delay = Offset + Random Delay.


43/72

Synchronizing Timer The purpose of the SyncTimer is to block threads until

X number of threads have been blocked, and then theyare all released at once. A SyncTimer can thus create

large instant loads at various points of the test plan.


44/72

Sync timer results


45/72

Synchronization timer(contd):

In the above slide Random timer is kept to ensure thatall the threads are delayed by random amount of time.Each thread is delayed by 3 to 13 seconds.

Synchronization timer is used to make sure that all thedesired no. of threads hit simultaneously at a link. So

that the response of the page can be retrieved. If anythread reaches to the page first it has to wait for thedesired no of threads , so that they all can gosimultaneously to the page.


46/72

Pre- Processors


47/72

A Pre-Processor executes some action prior to a SamplerRequest being made. If a Pre-Processor is attached to aSampler element, then it will execute just prior to thatsampler element running. A Pre-Processor is most often

used to modify the settings of a Sample Request justbefore it runs, or to update variables that aren't extractedfrom response text.

Some of the Pre Processors

1) HTML Link Parser2) Counters3) User Parameters

Pre-Processor


48/72

HTML Link ParserThis modifier parses HTML response from the serverand extracts links and forms. A URL test sample thatpasses through this modifier will be examined to see if

it "matches" any of the links or forms extracted fromthe immediately previous response. It would thenreplace the values in the URL test sample withappropriate values from the matching link or form


49/72


50/72

Post Processors


51/72

Post ProcessorsA Post-Processor executes some action after a SamplerRequest has been made. If a Post-Processor isattached to a Sampler element, then it will execute

just after that sampler element runs. A Post-Processoris most often used to process the response data, oftento extract values from it

1) Regular expression extractor

2) XPath Extractor


52/72

Regular expression extractor


53/72

ResultsRegular Expression Extractor


54/72

Assertions


55/72

Assertions

Assertions allow the ability to assert facts aboutresponses received from the server being tested.Using an assertion, you can essentially "test" your that

your application is returning the results you expect itto.

1) Response assertion

2) Size assertion

3) Duration assertion


56/72

Response Assertions

Response Assertion


57/72


58/72

Recording using Jmeter for http

request 1. start jmeter with jmeter.bat on windows and jmeter.sh on unix 2. select test plan on the tree 3. right click on the test plan and add a new thread group: add -> thread

group 4. select the thread group

5. right click add -> config element -> Http Request Defaults 6. Protocol enter HTTP 7. Server name enter jakarta.apache.org 8. Path leave blank 9. Port number enter 80 10.select workbench

11.right click on workbench and add the Http proxy: add -> non-testelements -> HttpProxy Server 12.Port field - Enter 8080 Now also set the Browser proxy settings as localhost and port 8080 then

start the test from Jmeter


59/72

Proxy Server


60/72

Browser proxy setting


61/72

Listeners


62/72

Aggregate Report


63/72

Parameters of Aggregate Report Label - The label of the sample. If "Include group name in label?" is selected, thenthe name of the thread group is added as a prefix. This allows identical labels fromdifferent thread groups to be collated separately if required.

# Samples - The number of samples for the URL

Average - The average time of a set of results

Median - The median is the time in the middle of a set of results. 50% of thesamples took no more than this time; the remainder took at least as long.

90% Line - 90% of the samples took no more than this time. The remainingsamples at least as long as this. (90 th percentile)

Min - The shortest time for the samples of the given URL

Max - The longest time for the samples of the given URL

Error % - Percent of requests with errors Throughput - Throughput measured in requests per second/minute/hour

Kb/sec - The throughput measured in Kilobytes per second

Times are in milliseonds.


64/72

Recording from Badboy for Https

requestRecording https request from badboy


65/72

Badboy test plan after importing to

jmeter


66/72

Remote Testing


67/72

Distributed Test It is recommended that we should not run more than 200threads from one machine, so in order to test for 600 or1000 users we go for distributed testing

Master the system running Jmeter GUI, which controlsthe test

Slave the system running jmeter-server, which takescommands from the GUI and send requests to the targetsystem(s)

Target the web server we plan to load or stress Test


68/72

Remote Testing concept Before we go ahead with distributed testing, there are acouple of things to check.

1. the firewalls on the systems are turned off.

2. all the clients are on the same subnet. 172.16.50.121,172.16.50.101..

3. the server is in the same subnet, if 192.x.x.x or 10.x.x.x ipaddresses are used. If the server doesn't use 192 or 10 ipaddress, there shouldn't be any problems.

4. Make sure jmeter can access the server. 5. Make sure you use the same version of Jmeter on all the

systems. Mixing versions may not work correctly.


69/72

Remote testing steps On the slave systems, go to jmeter/bin directory and execute jmeter-server.bat (jmeter- server on unix). On windows, you should see a dos window appear with jre\[version] \bin\rmiregistry.exe. If this doesn't happen, it means either the environment settings are not right, or there are multiple JRE installed on the system. Note: [version] would be the jre version installed on the system.

1. Open jmeter-server.bat in a text editor 2. go to line 44 and find :setCP 3. edit START rmiregistry to the full path. Example: START C:\j2sdk1.4.2\jre\bin\rmiregistry 2. On master system acting as the console, open windows explorer and go to jmeter/bin directory 3. open jmeter.properties in notepad or wordpad 4. edit the line remote_hosts=127.0.0.1 5. add the IP address. For example, if I have jmeter server running on 192.168.0.10, 11, 12, 13, and 14, the entry would like like this: remote_hosts=192.168.0.10,192.168.0.11,192.168.0.12,192.168.0.13,192.168.0.14 6. Start jmeter. 7. Open the test plan you want to use


70/72

Distributed Testing architecture


71/72

Remote testing


72/72

Remote testingMachine name is different under samplers

Documents

Performance Testing With Open Source Tool JMeter