How many times have we seen a test infrastructure and methodology where the team is not able to get early feedback about the performance of the system they are developing? Typically, it is expected to treat performance testing as a “waterfall project” where we, the testers, prepare a test and run it into production right before its deployment. However, there is a better way to do this and it is with continuous performance testing. When implemented correctly, it gives confidence to the developers and keeps them aware of any substantial degradation in the system’s performance.
As a response to the growing trend in performance testing tools and as our Chief Technology Officer, Roger Abelenda further explores in this article, we have developed an open-source solution we believe can make a huge contribution to the software engineering community. The project’s name is called “JMeterDSL” and we want to show you how you can benefit from it.
What is JMeterDSL?
JMeterDSL is a new Java API that harnesses coding benefits for creating and running JMeter tests. Its main purpose is to provide a programmer-friendly API that allows testers and developers to create more readable test plans with a git-friendly format. This is especially recommended for teams that use, or want to start using, load testing in the CI/CD environment. The application will help them scale the tests in a continuous flow.
For all testers, developers, and tech leaders out there who want to dig deeper, we strongly recommend you to watch this webinar. In it, Sofia Palamarchuk and Melissa Chawla discuss the main advantages of this approach and how it helped Shutterfly implement continuous performance testing.
Having JMeter tests as code presents several advantages such as making test scripts shorter and therefore faster to read, edit and maintain. It makes it easier to modularize the scripts, and has inline documentation available. JMeterDSL also offers a user’s guide that, besides introducing the DSL itself, aims to reduce JMeter’s learning curve by providing best practices, warnings and tips.
My first test with JMeterDSL, step by step
We will guide you through the creation of a simple test for the Opencart site, which visits the website and selects a product. By doing so, we will review the main features that you will need when scripting a performance test.
Setting up
In this example, we will use Maven. You can check out other options for the project’s set up in the user guide.
To start using the JMeterDSL, you only have to create a Maven project and include the following dependency in your pom.xml file.
<dependency>
<groupId>us.abstracta.jmeter</groupId>
<artifactId>jmeter-java-dsl</artifactId>
<version>0.28</version>
<scope>test</scope>
</dependency> We will also use JUnit 5 as a test library and AssertJ for test plan assertions.
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>RELEASE</version>
<scope>test</scope>
</dependency><dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.21.0</version>
<scope>test</scope>
</dependency> Recording the test flow
Typically in JMeter, we record our tests using the JMeter Recorder or a third-party proxy to capture the requests, as this allows us to see more details than JMeter’s Test Script Recorder. In this example we will use fiddler, you can download it here.
Once we have our proxy on, we can execute the steps manually in the browser and view the requests in fiddler.
Having deleted all requests made to other hosts, and static resources (.js, .css and image files), this is what the recording looks like:
The next step is to map these requests to JMeterDSL.
Creating the script
Let’s start with a simple script that executes one iteration of one user executing a GET request to the Opencart home page.
Just like in JMeter, the first step is to create a test plan in which we will add a thread group as a parameter. In the thread group, we can define the number of threads (virtual users), iterations, and samplers to be executed. To execute an HTTP request we use the httpSampler method with the desired URL.
import static us.abstracta.jmeter.javadsl.JmeterDsl.*;
import java.io.IOException;
import org.junit.jupiter.api.Test;
public class OpencartTest {
@Test
public void OpencartTest() throws IOException {
testPlan(
threadGroup(1, 1,
httpSampler("http://opencart.abstracta.us")
)
).run();
}
}Having done this, we can add an assertion to validate that the 99th percentile of the requests’ response time is less than 5 seconds. In JMeter, this would be impossible without installing the right plugins. (Check more about Understanding Key Performance Testing Metrics)
To do this, we need to save the test plan stats into a variable, so then we can use it to get different metrics of the execution. In this case, we will use sampleTimePercentile99, and the AssertJ method isLessThan.
import static org.assertj.core.api.Assertions.assertThat;
import static us.abstracta.jmeter.javadsl.JmeterDsl.*;
import java.io.IOException;
import java.time.Duration;
import org.junit.jupiter.api.Test;
import us.abstracta.jmeter.javadsl.core.TestPlanStats;
public class OpencartTest {
private final String protocol = "http://";
private final String host = "opencart.abstracta.us";
private final String baseUrl = protocol + host;
@Test
public void OpencartTest() throws IOException {
TestPlanStats stats = testPlan(
threadGroup(1, 1,
httpSampler(baseUrl)
)
).run();
assertThat(stats.overall().sampleTimePercentile99()).isLessThan(Duration.ofSeconds(5));
}
}Now when we run the test, if the 99th percentile of the response times is higher than or equal to 5 seconds, the test will fail. As you can see we have parameterized the base url and protocol to avoid duplication.
Adding headers
We have created the first get request, but it’s still missing the headers. To add them we can use the method header with the correct name and value.
Looking at the fiddler recording we can see all the headers that are sent along with this request.
To add them to the sampler we use the method header(“name”,”value”).
httpSampler(baseUrl)
.header("Host", host)
.header("Connection", "keep-alive")
.header("Upgrade-Insecure-Requests", "1")
.header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36")
.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9")
.header("Accept-Encoding", "gzip, deflate")
.header("Accept-Language", "en-US,en;q=0.9")Another option here is using constants. We can import HTTPConstants like this:
import org.apache.jmeter.protocol.http.util.HTTPConstants;And now we can use the constants for the headers names. For example, this is how the host header would look like:
.header(HTTPConstants.HEADER_HOST,host)Now let’s add the headers for all the recorded requests. Ultimately, the thread group should look like this:
threadGroup(1, 1,
httpSampler(baseUrl)
.header("Host", host)
.header("Connection", "keep-alive")
.header("Upgrade-Insecure-Requests", "1")
.header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36")
.header("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9")
.header("Accept-Encoding", "gzip, deflate")
.header("Accept-Language", "en-US,en;q=0.9"),
httpSampler(baseProductsUrl + productQuery)
.header("Host", host)
.header("Connection", "keep-alive")
.header("Upgrade-Insecure-Requests", "1")
.header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36")
.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9")
.header("Referer", baseUrl + "/")
.header("Accept-Encoding", "gzip, deflate")
.header("Accept-Language", "en-US,en;q=0.9"),
httpSampler(baseProductsUrl + "/review" + productQuery)
.header("Host", host)
.header("Connection", "keep-alive")
.header("Accept", "text/html, */*; q=0.01")
.header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36")
.header("X-Requested-With", "XMLHttpRequest")
.header("Referer", baseProductsUrl + productQuery)
.header("Accept-Encoding", "gzip, deflate")
.header("Accept-Language", "en-US,en;q=0.9")To tidy up the script we can group the last two requests in a transaction, as they correspond to the same action, and add names to each request, so we can identify these transactions/steps easily at execution time.
As you can see I have also reduced duplicated code by defining these variables:
String baseProductsUrl = baseUrl + "/index.php?route=product/product";
String productQuery = "&product_id=40";We can also use a shared httpHeaders element with headers that are repeated in every request. Like in JMeter, this element will affect all elements at the same level, so to achieve a shared httpHeaders we must create it directly in the thread group within the same scope as the rest of the samplers.
This is how the thread group looks now:
threadGroup(1, 1,
httpHeaders()
.header("Host", host)
.header("Connection", "keep-alive")
.header("Accept-Encoding", "gzip, deflate")
.header("Accept-Language", "en-US,en;q=0.9")
.header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"),
httpSampler("Enter Opencart website", baseUrl)
.header("Upgrade-Insecure-Requests", "1")
.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"),
transaction("Click product",
httpSampler("Click product - product", baseProductsUrl + productQuery)
.header("Upgrade-Insecure-Requests", "1")
.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9")
.header("Referer", baseUrl + "/"),
httpSampler("Click product - review", baseProductsUrl + "/review" + productQuery)
.header("Accept", "text/html, */*; q=0.01")
.header("X-Requested-With", "XMLHttpRequest")
.header("Referer", baseProductsUrl + productQuery)
)
)Adding response assertions
Validations, such as this one, confirm that the server is responding as expected and verifies that the script we are developing is acting as it should. To add assertions we will use the element responseAssertion. We can use methods like containsSubstrings, equalsToStrings, containsRegexes and matchesRegexes. To use one of them in a sampler we must add it as a child like this:
httpSampler("Enter Opencart website", baseUrl)
.children(
responseAssertion().containsSubstrings("<title>Your Store</title>")
)In the example above, we are asserting the presence of the title within the obtained response, using a portion of the page’s HTML.
Adding timers
Timers are important to properly mimic a real user’s behavior. For adding a timer we can use uniformRandomTimer(minimumMillis,maximumMillis). This method adds a pause between the selected values before executing the request. If we want the timer to affect only one sampler, we must add it as its child. Just like in JMeter, if we were to add the timer at the same level as the request, it would affect every request in the thread group.
httpSampler("Enter Opencart website", baseUrl)
.children(
uniformRandomTimer(1000, 2000)
)Extracting variables from the response
With the script as it is, we are always selecting the same product (product id) from the web pages in every run. To make this selection dynamic, let’s extract this product id from the previous response.
To do that, we will use regexExtractor(“variableName”,”regex”) as a child of the sampler. It works just like JMeter’s Regular Expression Extractor, so you can also modify other options like the template, default value, match number, and target field.
We will define a variable named “product”. This variable contains the name of the product that we will extract the ID from, and we will use it to create a regular expression.
String product = "iPhone";httpSampler("Enter Opencart website", baseUrl)
.children(
regexExtractor("productId", "product_id=(\\d*)\">" + product)
)Using extracted variables
To use the extracted value we can reference the variable using “${variableName}”. So, if we want to use the variable productId to select a product, we can embed it as shown below:
String productQuery = "&product_id=${productId}";Adding CSV files
Now, what if we want each user to select a different product? Let’s add a CSV file with 3 different products and modify the thread group to run 3 users.
To add the CSV we use csvDataSet(“csvFile”). We can use the full or relative path from the root of the project.
csvDataSet("./src/test/java/products.csv"),
threadGroup(3, 1,
httpSampler("Enter Opencart website", baseUrl)
.children(
responseAssertion().containsSubstrings("<title>Your Store</title>"),
regexExtractor("productId", "product_id=(\\d*)\">${product}"),
uniformRandomTimer(1000, 2000)
),
transaction("Click product",
httpSampler("Click product - product", baseProductsUrl + productQuery),
httpSampler("Click product review", baseProductsUrl + "/review" + productQuery)
)
)Debugging
For debugging you can add the element resultsTreeVisualizer() in the test plan. If we enable this option, JMeter’s built-in View Results Tree element will be displayed. This will allow us to view the transactions in real-time while running the script, displaying the requests and responses for each sample in addition to collected metrics.
Reporting
You can generate an html report by simply adding the element htmlReporter(“reportDirectory”) where reportDirectory has to be an empty directory. You can generate a new directory every time like this:
htmlReporter("html-report-" + Instant.now().toString().replace(":", "-"))Final Thoughts
We think JMeterDSL is a great tool for anyone, with or without JMeter and or programming experience. It significantly eases creation, execution and maintenance of performance tests. Whenever you are trying to create load tests in a quick fashion, validate your job, and or include them in a CI/CD pipeline, JMeterDSL will now be an option.
Now is the time to give JMeterDSL a try! Check it out and view the user guide for more information.
Help us with any feedback and become a contributor. We will take every comment into consideration: bugs, feature requests, adjustments, anything. We are open to co-create JMeterDSL to make it the best open source dev-friendly tool for performance testing.
We are really looking forward to hearing your thoughts and ideas. Please don’t hesitate to leave us a comment in the comment section below.
Follow us on Linkedin, Facebook, Twitter, and Instagram to be part of our community!
Tutorial: How to Use Gatling With Maven
Another key integration for enabling CI/CD In this post, we will see how to use Gatling with Maven, creating an integration via plugins. In the world of CI/CD, attaining this kind of integration has become essential. In particular, I’ll show you how to integrate Gatling, use…
New Release! Integration between JMeter DSL and Azure Load Testing
We have released a new feature in JMeter DSL that significantly simplifies the use of Azure Load Testing when scaling tests. This allows performance tests developed with JMeter DSL to be run on Azure Load Testing, complementing the existing integrations with BlazeMeter and OctoPerf. We…
Abstracta
