The moving target - Performance Testing / Engineering in real world

When hungry, feed is the norm. But if you feel hungry frequently, does it mean

that you have to feed more? Well, there may be a real problem in the system and

I think we need to get to root cause rather than feeding more.

This is exactly what has been happening in the system performance world. When

system goes slow or there are errors in accessing the system, normally, the

tendency is to add additional hardware rather than getting to the root cause.

So, lets look at what is performance testing?

Many would answer that it is performed to determine and measure the system

performance in a given workload condition.

Well… That is correct but!!!!

To answer BUT, lets look at what is a system today?

System today consists of various sub systems, which interact, integrate and work

together.

Lets take an example, I have an e-commerce application available on my mobile

to shop and the same is available on web browser as well. Tomorrow, it will be

available on TV and additional devices.

These systems are interacting with various other systems on backend such as

store management, warehouse management, delivery management, finance

management, payment gateways and more.

So what happens now?

Usually, the e-commerce company engages with a team of performance testers to

understand if the system can handle load in certain conditions.

There are 2 working conditions to any of the applications:

1. The best scenario which actually may not the best for the e-commerce

company – Usual load conditions

2. The worst scenario which may be the best for the e-commerce company –

Heavy load condition or spike in load

They get a set of performance reports, which provides information of system

behaviour under different conditions.

Well what next?

The response would normally be the front end application response time and not

other integrated server parameters and if the response time does not meet the

SLA criteria set, the normal recommendation would be to throw more hardware

to the problem. Customers sometime go overboard in adding additional

hardware which costs. This can sometimes solve the problem but will it on a long

run?

The answer to this is NO.

What are we missing?

1. Different applications behave differently when integrated

2. Each integrated application itself needs be optimised so that it does not

add additional load to the system

3. System / OS settings itself can help solve many problems if done right

4. Database settings needs to be optimised

5. The web servers sometimes add to the problem and needs to be configured

properly

Once all the settings, configurations and optimisations are in place, we can now

answer, how much is actually right.

So, today it is not merely performance testing but performance engineering. As

part of a performance engineering engagement, an organization should be able

to help:

1. Guide customers to the problematic areas

2. Coordinate with developers, System engineering, database engineers and

vendors to provide them insights of issues which can be corrected

3. Provide customers benchmark results and inputs on hardware required in

various conditions to help them plan and budget properly

4. Help them implement the right production strategy for a long haul rather

than a short term fix