Executing performance tests more effectively and efficiently

A thought by Scott Barber with which I fully agree reflects part of what I want to discuss here:

“Only performance testing at the conclusion of system or functional testing is like ordering a diagnostic blood test after the patient is dead.”

What we are interested in here at Abstracta, is how to execute a performance test most effectively and efficiently. 

It’s typical for performance tests to be left until the end of a project (provided there is still time and money left in the budget), and without having done any previous research. This means that all performance problems are encountered at the end and all at once. This is not the most effective approach to performance testing because as Barber puts it, it may be as useless as running diagnostics on a dead patient.

Instead of running tests as an afterthought, let’s consider what happens with functional tests. Most of the time, but unfortunately not always, unit tests are carried out by each developer prior to putting the system together. Once the system is integrated, integration tests are run and a new version is forwarded to an external testing team to perform the system testing as a whole.

We believe that unit testing, applied to performance and to many other non-functional aspects as well, could result in a much more refined application upon the final testing stage. This way, fewer problems will arise at the end of the project.

Unit Performance Testing

It is not hard to implement a program for executing a method several times and to record the response times. While that goes on, we could check the database, the timing of SQLs, and whether indices were used correctly. We could also consider what would happen if the table grew in size among other things.

This way, instead of just executing a method several times, that program could gather multiple processes to execute that method several times, always bearing in mind what will happen in production. We could measure the memory consumed, and how the connections pool is managed, or whether we will observe blocking among tables, etc.

Unit performance testing advantages

• Lower risk of encountering performance problems
• Problems are detected in advance, making solutions less expensive
• Developers are committed to performance and carry out verifications to avoid releasing modules with serious performance problems
• Developers become familiar with the use of their systems’ resources and unveil the things in the code that generate problems in order to avoid them in the future

Unit performance testing disadvantages

• More time is needed to release the first version
• One must learn how to use monitoring tools

Of course, this cannot be done with every method developed. However, we should take into account, for example, those most used and with the most demanding tasks in consumption, or those with more data processing, etc.
These unit performance tests are obviously no substitute for performance tests. They are a way of being better prepared for them. They’re not a substitution because the various functionalities are not tested together and no tests are run on the operations. Also, these tests are not carried out on a server similar to the production server, but rather on the development server.

While many developers see performance testing as an afterthought, it is important to perform smaller intermittent tests during production. This way, in the end, the final performance test will result in fewer problems, a smaller chance of having to make major adjustments, and will ultimately lower costs.

Recommended for You

Is the System Actually Running Slow?
How to Make a Performance Test Plan