{"id":11942,"date":"2026-03-16T23:21:03","date_gmt":"2026-03-16T23:21:03","guid":{"rendered":"http:\/\/abstracta.us\/blog\/?p=11942"},"modified":"2026-03-16T23:21:06","modified_gmt":"2026-03-16T23:21:06","slug":"performance-testing-metrics","status":"publish","type":"post","link":"https:\/\/abstracta.us\/blog\/performance-testing\/performance-testing-metrics\/","title":{"rendered":"Performance Testing Metrics: What to Measure and Why"},"content":{"rendered":"\n

Average response time, standard deviation, and p90, p95, and p99 are key performance testing metrics. Learn what these numbers are really<\/strong> telling you.<\/strong><\/p>\n\n\n\n

\"Ilustrative<\/figure>\n\n\n\n
\n

What if the biggest risk in performance testing isn\u2019t the code, but the way teams read the numbers?<\/strong>
Performance testing metrics that look fine on paper can still hide slowdowns, bottlenecks, and user frustration, as they define how people experience your system and affect users.<\/p>\n<\/blockquote>\n\n\n\n

Problems start when the development team do not fully understand what each metric shows. Wrong conclusions follow, leading to performance degradation instead of consistent performance.<\/p>\n\n\n\n

Misread your metrics, and stability is an illusion\u2014users experience significant delays before teams even realize what is happening.<\/p>\n\n\n\n

This post highlights average response time, standard deviation, and percentiles 90, 95, and 99, in detail, showing why these metrics matter for reliable growth<\/strong>. We also explore additional metrics that give a fuller view of system performance.<\/strong><\/p>\n\n\n\n

You only get one chance to perform. <\/strong>As systems grow, reading performance metrics well is only part of the challenge. Our performance testing services<\/a> help you detect bottlenecks early and validate system behavior before users are impacted.<\/p>\n\n\n\n

\n

<\/span>The Importance of Analyzing Data as a Graph<\/span><\/h2>\n<\/blockquote>\n\n\n\n

The first time we thought about this subject was during a course that Scott Barber<\/a> gave in 2008 (when we were just starting up Abstracta<\/a>), on his visit to Uruguay. He showed us a table with values like this:<\/p>\n\n\n\n

\"Performance<\/figure>\n\n\n\n

He asked us which data set we thought had the best performance, which is not quite as easy to discern as when you display the data in a graph:<\/p>\n\n\n\n

\"Performance<\/figure>\n\n\n\n

In Set A, you can tell there was a peak, but then it recovers.<\/p>\n\n\n\n

\"Performance<\/figure>\n\n\n\n

In Set B, it seems that it started out with a very poor response time, and probably 20 seconds into testing, the system collapsed and began to respond to an error page, which then got resolved in a second.<\/p>\n\n\n\n

\"Performance<\/figure>\n\n\n\n

Finally, in Set C, it\u2019s clear that as time passed, the system performance continued to degrade.<\/p>\n\n\n\n

Barber\u2019s aim with this exercise was to show that it\u2019s much easier to analyze information when it\u2019s presented in a graph.<\/strong> In addition, in the table, the information is summarized, but in the graphs, you can see all the points. Thus, with more data points, we can gain a clearer picture of what is going on.<\/p>\n\n\n\n

Interested in data analysis? Keep learning here: <\/strong>Data Observability: What It Is and Why It Matters<\/strong><\/a>.<\/strong><\/p>\n\n\n\n

<\/span>Understanding Key Performance Testing Metrics<\/span><\/h2>\n\n\n\n
\"Illustrative<\/a><\/figure>\n\n\n\n

Okay, now let\u2019s see what each of the metrics for performance testing means as a key part of your performance testing process. Evaluating key performance indicators helps confirm that your tests align with business objectives.<\/strong><\/p>\n\n\n\n

<\/span>Average Response Time<\/span><\/h3>\n\n\n\n

Average response time is the mean time a system takes to respond to a request during performance testing. To calculate the average, simply add up all the values of the samples and then divide that number by the quantity of samples.<\/strong><\/p>\n\n\n\n

Let\u2019s say we do this, and our resulting average peak response time is 3 seconds. The problem with this is that, at face value, it gives you a false sense that all response times are about three seconds, some a little more, and some a little less, but that might not be the case.<\/p>\n\n\n\n

Imagine we had three samples, the first two with a response time of one second, the third with a response time of seven:<\/p>\n\n\n\n

1 + 1 + 7 = 9<\/p>\n\n\n\n

9\/3 = 3<\/p>\n\n\n\n

This is a very simple example that shows that three very different values could result in an average of three, yet the individual values may not be anywhere close to 3.<\/p>\n\n\n\n

Fabian Baptista<\/a>, co-founder and Chief Innovation Officer at Abstracta, made a funny comment related to this:<\/p>\n\n\n\n

\n

\u201cIf I were to put one hand in a bucket of water at -100 degrees Fahrenheit and another hand in a bucket of burning lava, on average, my hand temperature would be fine, but I’d lose both of my hands.\u201d <\/em><\/p>\n<\/blockquote>\n\n\n\n

So, when analyzing average response time, it’s possible to have a result that’s within the acceptable level, but be careful with the conclusions you reach.<\/p>\n\n\n\n

That’s why it is not recommended to define service level agreements (SLAs) using averages<\/strong>; instead, have something like \u201cThe service must respond in less than 1 second for 99% of cases.\u201d<\/em> We’ll see more about this later with the percentile metric.<\/p>\n\n\n\n

Don’t miss this Quality Sense Podcast episode<\/a> about why observability is such relevant in software testing, with Federico Toledo<\/a> and Lisa Crispin<\/a>.<\/p>\n\n\n\n

<\/span>Standard Deviation<\/span><\/h3>\n\n\n\n

Standard deviation is a performance testing metric that shows how much response times vary around the average. <\/strong>If the value of the standard deviation is small, this indicates that all the values of the samples are close to the average, but if it\u2019s large, then they are far apart and have a greater range.<\/p>\n\n\n\n

To understand how to interpret this value, let\u2019s look at a couple of examples.<\/strong><\/p>\n\n\n\n

If all the values are equal, then the standard deviation is 0. If there are very scattered values, for example, consider 9 samples with values from 1 to 9 (1, 2, 3, 4, 5, 6, 7, 8, 9), the standard deviation is ~ 2.6 (you can use this online calculator<\/a> to calculate it).<\/p>\n\n\n\n

Although the value of the average as a metric can be greatly improved by also including the standard deviation, what’s more useful yet are the percentile values.<\/p>\n\n\n\n

Understanding key performance testing metrics is the first step.
<\/strong>Let\u2019s apply them together to deliver measurable business value.
<\/strong>Book a meeting<\/strong><\/a><\/p>\n\n\n\n

<\/span>Percentiles: p90, p95, and p99<\/span><\/h2>\n\n\n\n
\"Illustrative<\/a><\/figure>\n\n\n\n

Percentiles in performance testing are metrics that show the value below which a certain percentage of response times fall.<\/strong><\/p>\n\n\n\n

Understanding them is crucial for accurate analysis during test execution, since they reflect how systems behave when many users interact simultaneously. Percentiles reveal performance under varying loads, helping teams identify bottlenecks and optimize resource allocation.<\/p>\n\n\n\n

Let\u2019s break down what percentiles like the 90th percentile (p90), p95, and p99 mean and how they can be used effectively in performance tests.<\/strong><\/p>\n\n\n\n

<\/span>What Are Percentiles?<\/span><\/h3>\n\n\n\n

A percentile is a performance testing metric that shows the value below which a certain percentage of results fall. It reveals how response times are distributed across all samples. The percentile rank is another metric that indicates the relative position of a given value within that distribution.<\/p>\n\n\n\n

The 90th Percentile (p90)<\/h4>\n\n\n\n

The 90th percentile (p90) indicates that 90% of the sample values are below this threshold, while the remaining 10% are higher. This is useful for identifying the majority of user experiences and boosting that most users have acceptable response times.<\/p>\n\n\n\n

The 95th Percentile (p95)<\/h4>\n\n\n\n

The 95th percentile (p95) shows that 95% of the sample values fall below this threshold, with only 5% above it. This provides a more stringent measure of performance, enabling nearly all users to have a good experience.<\/p>\n\n\n\n

The 99th Percentile (p99)<\/h4>\n\n\n\n

The 99th percentile (p99) represents the value below which 99% of the sample falls, leaving just 1% as outliers. This is particularly valuable for identifying outliers and making it possible that even the worst-case scenarios are within acceptable limits.<\/p>\n\n\n\n

<\/span>Why Use Multiple Percentiles?<\/span><\/h3>\n\n\n\n

Analyzing multiple percentile values, such as p90, p95, and p99, provides a more detailed view of system performance. <\/strong>Tools like JMeter and Gatling include these in detailed reports, allowing teams to calculate percentile scores using different methods. <\/p>\n\n\n\n

This comprehensive approach helps in identifying performance bottlenecks and understanding how the system behaves under various conditions. It can also reveal whether issues are tied to backend processing, traffic spikes, or network latency.<\/p>\n\n\n\n

<\/span>Complementing Percentiles with Minimum, Maximum, and Median Metrics<\/span><\/h3>\n\n\n\n

Percentiles provide more context when combined with metrics like minimum, maximum, and median values. For example:<\/p>\n\n\n\n