Basics of Performance Testing

By

PHANI BHUSHAN NAGULA ISTQB Adv TM, ISTQB CTFL, ISTQB CAT, TMMI-P,

CSM, PRINCE2 Practitioner, SSGB, ITIL V3 Foundation

1.0 Basics of Performance Testing

Syllabus

Basics of Performance Testing

What is Performance Testing?

Purpose of Performance Testing?

Types & Benefits of Performance Testing

Risks Addressed by Performance Testing

Performance Test Metrics

Performance Testing Maturity

Performance Testing Tool Selection

Designing Performance Test Environment

Performance Test Process Flow

What is Performance Testing?

A well-performing application is one that lets the end user carry out a

given task without undue perceived delay or irritation. Performance is

really is in the eye of the beholder.

- The End User Perspective

Performance testing is the process of determining the speed or

effectiveness of a computer, network, software program or device.

Performance Testing is intended to determine responsiveness, throughput,

reliability, and/or scalability of a system under a given workload.

Performance Testing is conducted to accomplish the following:

Assess production readiness

Assessing infrastructure adequacy

Assessing adequacy of developed software performance

Improving the efficiency of performance tuning

Purpose of Performance Testing?


Type Benefits

Performance Testing

Determines the speed, scalability and stability characteristics of an application, thereby providing an input to making sound business decisions.

Focuses on determining if the user of the system will be satisfied with the performance characteristics of the application.

Identifies mismatches between performance-related expectations and reality.

Supports tuning, capacity planning, and optimization efforts.

Load Testing

To determine the application behavior under anticipated work load.

Determines the adequacy of a hardware environment.

Evaluates the adequacy of a load balancer.

Detects concurrency issues.

Detects functionality errors under load.

Type Benefits

Load Testing

Collects data for scalability and capacity-planning purposes.

Helps to determine how many users the application can handle before performance is compromised.

Helps to determine how much load the hardware can handle before resource utilization limits are exceeded.

Stress Testing

To determine or validate an application’s behavior when it pushed beyond normal or anticipated load conditions.

Determines if data can be corrupted by overstressing the system.

Provides an estimate of how far beyond the target load an application can go before causing failures and errors in addition to slowness.

Allows you to establish application-monitoring triggers to warn of impending failures.

Ensures that security vulnerabilities are not opened up by stressful conditions.

Helps to determine what kinds of failures are most valuable to plan for.


Other Types

Type Purpose

Capacity Test To determine how many users and/or transactions a

given system will support and still meet performance goals.

Endurance Testing Testing a system with a significant load extended over a

significant period of time, to discover how the system behaves under sustained use.

Spike Testing Verify a system's stability during bursts of concurrent

user and or system activity to varying degrees of load over varying time periods.

Stability

It refers to the overall reliability, robustness, functional and data integrity, availability, and/or consistency of responsiveness for your system under a variety conditions.

Risks Addressed by Performance Testing

Type Purpose

Load Testing

How many users can the application handle before undesirable behavior occurs when the application is subjected to a particular workload?

How much data can my database/file server handle? Are the network components adequate?

Stress Testing

What happens if the production load exceeds the anticipated load?

What indicators should we look for in order to intervene prior to failure.

Capacity Test Is system capacity meeting business volume under both

normal and peak load conditions?

Endurance Testing

Will application performance be consistent over time? Are there slowly growing problems that have not yet

been detected? Are there external interference that was not accounted

for?

Spike Testing

What happens if the production load exceeds the anticipated peak load?

What kinds of failures should we plan for? What indicators should we look for?

Performance Testing Maturity

Fire Fighting

Performance Validation

Performance Driven

Appropriate Tool Selection

Category Description

Test Tool Architecture

Scripting Module Test Management Module Load Injectors (multiple workstations / geographical

locations) Analysis Module

Feasibility Test tool should be compatible with application

technology.

Tool Vendor Support Official support?

Licensing Model No. of Virtual Users Application technology that tool can support Supported Protocols

Scripting Effort Little or no scripting effort? Skill level of testing team? Training support?

Appropriate Tool Selection


Injection Capacity

Ensure there are sufficient hardware resources to generate the required load.

IP Spoofing: where multiple IP addresses are allocated to the NIC on each injector machine and the tool allocates a different IP address to each virtual user.

Designing Performance Test Environment


Number and Specification of Servers

It is often impractical to provide an exact replica of live server specifications and architecture in test environment.

Try to match the live server specifications, if not number of servers.

Bandwidth and Connectivity of Network Infrastructure

It is rare for the target servers to be deployed in the same location as their local counterparts.

No. of Application Tiers Number of application tiers has greater impact on the

validity of performance testing than number of servers.

Sizing of Application Database

Size of test environment database should be comparable to live one.


Metric Description A

PP

LI

CA

TI

ON

Response Time (Min, Max & Avg)

Measures the performance of an individual transaction or query and is the amount of time from the moment that a user sends a request until the time that the application indicates that the request has completed.

Throughput It indicates the number of transactions per second an application can handle.

Latency A synonym for delay. Latency is a measure of responsiveness that represents the time it takes to complete the execution of a request.

90% Line 90% Line (90 percentile of the response times)represents one of the most interesting metrics and shows that “90% of the samples finished within 0-x time” or “90% of the samples took no more than this time”


Metric Description A

PP

LI

CA

TI

ON

Standard Deviation Is a measure of the variability of a data set. JMeter calculates the population standard

deviation, not the sample standard deviation .


Metric Description S

ER

VE

R

CPU Usage This metric provides information on machine CPU utilization. For effective application performance CPU utilization should not be more than 70%.

Memory Usage Memory usage provides the information on amount of free disk space, virtual memory pages moved between main memory and disk storage and percent memory used during the scenario execution.

Disk Space Usage This metric provides the information on percent disk space is used.


Metric Description N

ET

WO

RK

Network output queue length

length of the output packet queue in packets. Anything more than two means a delay and bottlenecking needs to be stopped.

Network bytes total per second

rate which bytes are sent and received on the interface including framing characters.

Performance Testing Process Flow

Technology

Basics of Performance Testing