Futurology: noun, systematically forecasting the future, especially from present trends in society

I’m not going to lie, I can’t predict the future. I don’t even know what I am going to eat for dinner most days, but in testing, sometimes we have to put our best hypotheses about the future to use. When designing performance tests, specifically a load test (read about the different types of performance tests), we should simulate a load that is as close to the real one as possible that the system will face, thereby reducing risk in production.

To design these tests it’s usually important to do some background research by conducting interviews with analysts, business experts, developers, infrastructure managers, etc. to get a feel for the expected load of a system. Often, you can look at the usage statistics (meaning, analyzing the access log of the system) if it is already in production. With this information you can generally obtain the load that the system is currently used to experiencing. You can seek out when it’s under the greatest stress and then simulate those moments.  

We have seen in some cases the importance of guiding the interviewee to think not only about the present load situation, but also to think about the future. It should be emphasized when designing performance tests to not only base the load scenario on access logs, but also to take into account the projected load.

A manager may read a performance report and say “Ok, with the load that we simulated, it seems that we won’t have any problems with effectively supporting a load like last month’s… but what if I do a marketing campaign and the site visits double?”

This is Where Futurology Comes In

In certain situations, you can directly establish the expected load scenario (also known as the 100% load scenario) as the load that you project for the future. Another way is to define our scenario of 100% as 100% of the current production today, and then run a test target for 110% or 150%, according to how we deem it will grow. In this way, we systematically are forecasting the future, or say, using futurology!

This clearly affects the effort estimated for the test. When we estimate the time spent on the executions in general we think of reaching 100% of the load and if we go into “over time,” we run tests with a larger load. But, it would be desirable to include in the same time schedule, test executions at perhaps 150% and 200% of the actual load for example, or according to the projections that exist.

How Do We Know the Projected Load? 

Is there some kind of crystal ball that will help us to determine the projected load?

Unfortunately, no. This is one of the major performance testing difficulties: estimating growth. As futurologists, we can analyze the historical growth rates and use them as a predictor of future growth or involve other people inside the company, like the marketing team, for example, who are the ones with a clear growth target and a plan for achieving it.

Another major difficulty of these tests usually has to do with the database used. For example, if we define the load as the user activity three years into the future, considering the expected growth, but the volume of data in the database used doesn’t contain the historical data that would result after three years of using the system, we would not be simulating the complete situation that the server would be exposed to.

Keep in mind the task of generating such data, or else make it clear that the test does not include that aspect, which may leave some uncertainty, but as always, one aims to maximize the cost/benefit ratio. So, if the cost of generating such data is very high, it might be preferable not to.

Of course, since designing tests takes time and resources, it’s important to be strategic. Ideally one would want to know at the very least, the maximum load that the system could possibly support, (independent of the client’s expectations) simply to document the point at which the system breaks or to discover potential problems beforehand.

How do you estimate the future expected load when designing performance tests?

Recommended for You

Today’s New Challenges for Performance Engineers and How to Deal with Them
Software Performance Testing Fallacies Part 1