13 Performance Testing Methodologyminisites.qaiglobalservices.com/stc2012/Paper_ Best_Practice/13... · Performance Testing Methodology ... Performance, Load, Stress, Endurance Testing

Smart Smart Testing ... Robust Results Testing ... Robust Results

PerformancePerformance Testing MethodologyTesting Methodology

Author : Anamica AbrolAuthor : Anamica Abrol

Testing ... Robust Results Testing ... Robust Results

Testing MethodologyTesting Methodology

Performance Engineering – An Overview

Performance Testing Methodology

What Performance Problems May Arise?

Major parameters to consider

ContentsContents

Baseline and Benchmarking

Relationship between Performance testing and tuning

Performance, Load, Stress, Endurance Testing

Benefits of Key Performance Test Types

An Overview

Performance Testing Methodology

What Performance Problems May Arise?

© InterGlobe Technologies Pvt. Ltd. 2011. Confidential: Not for Distribution. www.igt.in

Relationship between Performance testing and tuning

erformance, Load, Stress, Endurance Testing

Benefits of Key Performance Test Types

Performance EngineeringPerformance Engineering

An iterative process by which software is tested and tuned with the intent of realizing the required performance.

The concept of coupling performance testing with call it performance engineering.

The performance testing part of it encomapsses what's commonaly referred to as load, stress, enduarnce testing , as well as validating system perfromance.

It is all about solving a problem to achieve a desired and beneficial outcome.

Performance Engineering in its simplest terms

An iterative process by which software is tested and tuned with the intent of

testing with tuning has caught on, and now we

The performance testing part of it encomapsses what's commonaly referred to as load, stress, enduarnce testing , as well as validating system perfromance.

a problem to achieve a desired and beneficial outcome.

© InterGlobe Technologies Pvt. Ltd. 2011. Confidential: Not for Distribution. www.igt.in3

Phases of Performance Engineering

3

Phase Deliverable

1- Project Assessment Assessment Report

2 - Planning Test plan

3 - Scripting Test scripts

PerformancePerformance TestingTesting MethodologyMethodology

Proper Methodology guarantees a successful performance test project

4 - Test execution Test scenarios, test results

5 - Results analysis Results summary

6 - Reporting Performance test report and presentation

“Following a sound strategy in the engineering effort will increase your performance engineering team’s efficiency and effectiveness”.

- Scott Barber, www.pertesftestplus.comScott Barber, www.pertesftestplus.com

MethodologyMethodology

Proper Methodology guarantees a successful performance test project.


“Following a sound strategy in the engineering effort will increase your performance engineering team’s efficiency and effectiveness”.

Scott Barber, www.pertesftestplus.comScott Barber, www.pertesftestplus.com

Memory-related problems:

� Application uses more memory than it should.

� Memory leaks, Excessive garbage allocation.

Code-related problems:

� Application algorithms are not optimal.

Hardware/Software related Problem:

� Hardware you select for your database is not in vendor Hardware Compatibility List.

Major Performance Parameters to considerMajor Performance Parameters to consider

What Performance Problems May Arise?What Performance Problems May Arise?

Throughput :The number of units of work that can b

Latency: The time taken in establishing the network h

the time it takes to complete the execution of a request.

Response time: It is a measure of how responsive an application or subsystem is to a client. request.

Longevity: Execute tests for a pre-determined time interval with a pre

Hardware you select for your database is not in vendor Hardware Compatibility List.

Major Performance Parameters to considerMajor Performance Parameters to consider

What Performance Problems May Arise?What Performance Problems May Arise?


be handled per unit of time; for instance, requests per second.

k handshake or is a measure of responsiveness that represents

It is a measure of how responsive an application or subsystem is to a client. request.

determined time interval with a pre-determined load.

Baseline/BBaseline/Beenchmarkingnchmarking

The aspect from where the test execution actually begins.

Baselines are the metrics that can help identify changes in performance.

The intention here is two fold i.e.

All scripts need to be executed, validated, and debugged (if necessary) collectively.

Baseline and benchmark tests need to be conducted to provide a basis of comparison for all future testing.

“Assuming no performance problems are noticed during the scenario, the“Assuming no performance problems are noticed during the scenario, the

results obtained can be used as "best case" results. These results indicate how the system

performs when it is not under noticeable stress, but is still performing all required

functions”.

“If benchmark results do not meet the stated performance acceptance criteria, the

Performance Engineering Team and Stakeholder will work together to either resolve the

bottlenecks or revise the test strategy. Continuing on to the next phase of testing without

fixing the performance issues will not add value to the project”.

The aspect from where the test execution actually begins.

Baselines are the metrics that can help identify changes in performance.

All scripts need to be executed, validated, and debugged (if necessary) collectively.

Baseline and benchmark tests need to be conducted to provide a basis of comparison for all future

Assuming no performance problems are noticed during the scenario, the


Assuming no performance problems are noticed during the scenario, the

results obtained can be used as "best case" results. These results indicate how the system

performs when it is not under noticeable stress, but is still performing all required

“If benchmark results do not meet the stated performance acceptance criteria, the

Performance Engineering Team and Stakeholder will work together to either resolve the

bottlenecks or revise the test strategy. Continuing on to the next phase of testing without

fixing the performance issues will not add value to the project”.

Relationship between Perfomance Testing &Tuning Relationship between Perfomance Testing &Tuning

A more of cooperative effort between ProduSystem and Network Administrators.

The performance tester, or performance testing team, is a critical component as tuning typically requires additional monitoring of components, resources, and response times under a variety of load conditions and configurations.

Tuning follows an iterative process that is uperformance testing approach a project is following.

The cooperative testing and tuning team is test environment in order to maximize the effectiveness of the tuning phase.test environment in order to maximize the effectiveness of the tuning phase.

Perf tests are executes/re-executed after each change to the test environment, in order to measure the impact of a remedial change.

When a tuning phase is complete, the performance test should then be repeated to prove that the correct changes have been identified

Relationship between Perfomance Testing &Tuning Relationship between Perfomance Testing &Tuning

duct vendors, Architects, Developers, Testers, DB,

The performance tester, or performance testing team, is a critical component as tuning typically requires additional monitoring of components, resources, and response times under a variety of load conditions and configurations.

s usually separate from, but not independent of, the performance testing approach a project is following.

is generally given full and exclusive control over the test environment in order to maximize the effectiveness of the tuning phase.


test environment in order to maximize the effectiveness of the tuning phase.

executed after each change to the test environment, in order to

When a tuning phase is complete, the performance test should then be repeated to prove that the correct changes have been identified.

Performance, Load, Stress, Endurance TestingPerformance, Load, Stress, Endurance Testing

Performance can be classified into three main

Speed – Does the application respond quickly enough for the intended users

Scalability – Will the application handle the expected user load and beyond

Stability – Is the application stable under expected and unexpected user loads

Performance testing:

To determine or assess speed, scalability, and/or stability characteristics of an application; a parent category that includes load, stress, and endurance/duration testing.

A discipline concerned with determining and reporting the current performance of a software

application under various parameters.

Load testing:

To assess performance characteristics of an application experiencing anticipated load conditions.

Stress testing:

To find failure conditions or break points of an application that (hopefully) only occur under stressful conditions.

Endurance/Duration testing:

Load tests that run for a long time to detect emergent behaviours isn’t uncommon.

Performance, Load, Stress, Endurance TestingPerformance, Load, Stress, Endurance Testing

three main categories:

Does the application respond quickly enough for the intended users?

Will the application handle the expected user load and beyond?

Is the application stable under expected and unexpected user loads?

To determine or assess speed, scalability, and/or stability characteristics of an application; a parent category that includes load, stress, and endurance/duration testing.

A discipline concerned with determining and reporting the current performance of a software


To assess performance characteristics of an application experiencing anticipated load

To find failure conditions or break points of an application that (hopefully) only occur under

Load tests that run for a long time to detect emergent behaviours — 12 to 24 or even 72 hours

Benefits of Key Performance Test TypesBenefits of Key Performance Test Types

TermTerm BenefitsBenefits

Performance

test

�Determines the speed, scalability and stability characteristics of an application, thereby

providing an input to making sound business decisions.

�Focuses on determining if the user of the system will be satisfied with the performance

characteristics of the application.

�Identifies mismatches between performance-

�Supports tuning, capacity planning, and optimization efforts.

Load test �Determines the throughput required to support the anticipated peak production load.

�Determines the adequacy of a hardware environment.�Determines the adequacy of a hardware environment.

�Evaluates the adequacy of a load balancer.

�Detects functionality errors under load.

�Collects data for scalability and capacity-planning purposes.

�To determine how many users the application can handle before performance is

compromised.

�To determine how much load the hardware can handle before resource utilization limits are

exceeded.

Stress Test �Determines if data can be corrupted by overstressing the system.

�Allows you to establish application-monitoring triggers to warn of impending failures.

�Ensures that security vulnerabilities are not opened up by stressful conditions.

�Determines the side effects of common hardware or supporting application failures.

�Helps to determine what kinds of failures are most valuable to plan for.

Benefits of Key Performance Test TypesBenefits of Key Performance Test Types

Determines the speed, scalability and stability characteristics of an application, thereby

providing an input to making sound business decisions.

Focuses on determining if the user of the system will be satisfied with the performance

- related expectations.

Supports tuning, capacity planning, and optimization efforts.

Determines the throughput required to support the anticipated peak production load.

Determines the adequacy of a hardware environment.


Determines the adequacy of a hardware environment.

planning purposes.

how many users the application can handle before performance is

To determine how much load the hardware can handle before resource utilization limits are

Determines if data can be corrupted by overstressing the system.

monitoring triggers to warn of impending failures.

Ensures that security vulnerabilities are not opened up by stressful conditions.

Determines the side effects of common hardware or supporting application failures.

Helps to determine what kinds of failures are most valuable to plan for.

Thank You!! Thank You!! Thank You!! Thank You!!

© InterGlobe Technologies Pvt. Ltd. 2011. Confidential: Not for Distribution. www.igt.in 10

Documents

13 Performance Testing Methodologyminisites.qaiglobalservices.com/stc2012/Paper_ Best_Practice/13... · Performance Testing Methodology ... Performance, Load, Stress, Endurance Testing