Can you run performance tests in production?
Imagine that we have to run a test that we’ve already designed with a good load scenario, test cases, test data, all automated and prepared in the load simulation tool that we want to use, BUT we don’t have a testing infrastructure that compares to that of production. This usually happens when the production infrastructure is a large server, such as an AS400 or any other mainframe, a cluster composed of multiple machines, or a complex configuration, making it expensive to prepare an exclusive testing environment.
Where Should We Execute the Performance Test?
Surely the development server, or the one intended for functional testing, may not have the features that the production server has. What are our options in this case?
Well, the best option is to do performance testing in production!
We’ve been asked many times if you can run performance testing in production. What is the problem? Well, we assume many risks. What if you run a test of this kind on a system but then it affects the other systems that are in production? More we’ve been told that the AS400 does not fail, but this is not so, as it has happened to us more than once. That’s the main risk.
Moreover, a major problem is that the results we get are impacted by other applications that are in production on the same server.
However, this approach has a great advantage: we ensure that the test infrastructure is similar to that of production because they’re exactly the same! In this, we are including everything that relates to hardware, database software (including versions of each component, their configuration, etc.) and other systems that have to coexist in production (from antivirus software to the company’s other information systems).
Therefore, what we thought was a serious problem, is actually the ideal situation!
Conclusion
Do initial performance testing on another machine wherever you can to minimize the risk of finding serious problems that affect the performance of applications that are already in production. Try to solve the problems arising there under the load and then move to production. Whoever controls the test must have looked over the basic indicators: CPU, memory, disk and network, and be clear about what thresholds are no longer acceptable. Once those levels are reached, stop the test.
We could also consider running performance tests at times when there is less of a load, like at night or during weekends. The problem with this is that more than one person is needed to operate the simulation tool. If you really want to get good results from a performance test, it is necessary to have experts in infrastructure, someone from the development team, etc.
Have you faced this situation before? What approach did you take?
Recommended for You
Why Performance Testing is Necessary
How to Make a Performance Test Plan
k6 Review: Open Source Load Testing Tool for Developers
Have you heard the hype around k6 yet? Yes, yet another open source performance testing tool is making waves. In this post, I’ll share my k6 review in the hopes that it may help you decide if you should also pay attention to it. Overview…
Workload or Load Scenario in Performance Testing
In load tests we simulate the workload that an application will have in production, counting the concurrent users accessing the application, the tests cases that will be executed, and the frequency of executions by users, among other things. The concept of workload or load scenario…
Abstracta
