With all of the free and enterprise tools available for performance testing, there’s no excuse for having a system failure
MAY 2020 UPDATE– We’ve added some more tools to our original list: Apptim and Artillery 🙂
Performance tests reveal how a system behaves and responds during various situations. A system may run very well with only 1,000 concurrent users, but how would it run with 100,000? In terms of performance, we wish to achieve three main goals: speed, scalability, and stability.
So, to understand how your site or app behaves when 100 people access it at once, do you call all of your friends and have them all enter at the exact same time? That could be one way to run a load test… Or, you can use a tool specifically designed to simulate that kind of load and much more!
Here, we’ve made a list of our own favorite performance/load testing and application performance monitoring tools to help you build and maintain a highly reliable and performant system (In an unbiased, alphabetical order!). Several of these are open source and most of the paid ones offer free trials.
1. AgileLoad
AgileLoad is an enterprise-class performance testing solution for optimizing the performance of your web applications. AgileLoad promises to increase your ROI and productivity, allowing you to perform your tests faster while providing a deep analysis of test results and full support for all new technologies. It successfully gives in-depth performance diagnostics for a wide range of applications from those using the most advanced web2.0 technologies to more classic client/server or heavy clients.
Key Features
Intuitive graphic interface lets you capture and design sophisticated business scenarios
Automatic detection and analysis of performance issues
Highly customizable scripting and reporting engine
Support performance testing across the application lifecycle
Test against a broad range of applications and protocols
Record and replay RIAs using a variety of Web 2.0 technologies
Measurable and repeatable load generation with system and end user monitoring
Automatic recording of test scenarios
Simulate realistic user load with Agileload load injection engine
Runs under Microsoft Windows (2003 and higher) and is compatible with all web browsers
Pricing
Free version: Build all your test scenarios with 10 virtual users
Pay as you go: Power up your Agileload by renting from 50 to 10000 virtual users
What Makes it Unique?
The free version or “starter kit” allows for quick ramp-up use of the tool
Automatic modeling vastly reduces scripting time for subsequent test cases against the same app
End-to-end diagnostic features: The tool not only monitors the front-end, but also the back-end to detect the reason for performance degradations
Anomaly detection and suggestion features help quickly identify and resolve performance bottlenecks
Configurable reports, for a mix of audiences
2. Apache JMeter
The de facto standard and “king” of load testing tools, Apache JMeter is an open source, 100% java based application with a graphical interface. It is designed to analyze and measure the performance and load functional behavior of web applications and a variety of services.
JMeter allows you to perform various testing activities like performance, load, stress, regression and functional testing, in order to get accurate performance metrics against your web server. The tool supports variable parameterization, assertions (response validation), per-thread cookies, configuration variables and a variety of reports. JMeter users heavily rely on its various plugins as most of its “out of the box” features are implemented through them.
Key Features
User-friendly GUI: JMeter comes with a simple and interactive GUI
Platform independent: JMeter is written and developed using java, so it can run on any environment or workstation that accepts a Java virtual machine, for example – Windows, Linux, Mac, etc.
Supports various server types: Web (HTTP, HTTPS, SOAP), database (JDBC, LDAP, JMS) and mail (POP3)
Supports multiple protocols such as HTTP, JDBC, LDAP, SOAP, JMS, and FTP
Simulates multiple users by using virtual users or unique users in order to generate heavy load against web application under test
Multi-threading framework allows concurrent and simultaneous sampling of different functions by many or separate thread groups
Remote distributed testing: JMeter uses a Master-Slave concept for distributed testing where the master will distribute tests among all the slaves and the slaves will execute scripts against your server
Test results can be viewed in different formats like graph, table, tree, and report etc.
Pricing
Free / Open Source
What Makes it Unique?
JMeter has a very large community around it with endless resources and integrates with almost every performance testing tool on the market
Easy installation: It can be installed on any desktop with Windows, Mac or Linux
The test IDE allows test recording from browsers or native applications
Readily available plugins, for example, visualization plugin for data analysis
3. Appdynamics
AppDynamics is a leading tool that helps analyze, optimize and predict bottlenecks in complex information systems. It can be done on site, during product test and deployment and in the cloud. It has agents for .Net, node js, PHP, Java, C++, Python, Apache Web Server and GoLang. It allows not only real-time application performance monitoring, but business performance monitoring and end user monitoring. According to AppDynamics, its platform helps you to connect application performance and customer experience to your business outcomes with a smarter approach to performance management.
Key Features
Code level visibility
Dynamic baselining and powerful alerting
Application performance management agents
Data retention
Monitoring of multiple platforms
Troubleshoot performance issues in production
Trend database performance over time
Continuously monitor in a high-volume environment
Install and begin using immediately
Database agents
Server Visibility
Synthetic Monitoring
Mobile Real-User Monitoring
Browser Real-User Monitoring
Pricing
There are 3 tiers: APM Pro, APM Advanced, and APM Peak – contact the vendor for pricing
What Makes it Unique?
Ensures that businesses get useful information with its unified analytics functionality.
AppDynamics features an innovative approach called Unified Monitoring that incorporates various AppDynamics applications into a single robust solution that helps to determine the precise origin of any performance issue
4. Apptim
Apptim was co-founded by an Abstracta co-founder, Fabian Baptista, and Sofia Palamarchuk, our former CEO. The vision behind it is to make autonomous mobile testing a reality. The tool has been in beta until now. Today it offers an easy way for developers and testers to quickly gain an understanding of their mobile app’s performance, usability, and functionality.
Key Features
Test your mobile app on real devices
Supports manual and exploratory testing
Mobile performance dashboard with key metrics and rich data around resource usage, power consumption, start times, crashes, etc.
Integrates with Jira
iOS and Android testing
Performance troubleshooting and tuning
Pricing
Apptim is free – enterprise solutions available upon request.
What Makes it Unique?
This tool makes it effortless to fully test your mobile app in every build. Its biggest strength lies in its ability to help you to easily troubleshoot your app for crashes and exceptions while giving you full visibility on app performance trends.
5. Artillery
Artillery is a load testing and functional testing toolkit written in Node.js. It’s main purpose is to simulate realistic load on complex applications, it works with the concepts of virtual users that arrive to use the application in phases. You write your load testing scripts in YAML, with the option to use Javascript to write custom testing logic.
Key Features
Works from the command line
Easy integration with additional software for working with continuous integration
Less resources consumption while running
Works with HTTP(S), Socket.io, WebSockets, and AWS Kinesis
Simple JSON reports that can be exported and shared
Lets you create various scenarios and add them a weight property, the scenarios with higher weight have higher chances to be picked by the virtual users
Can set success conditions for latency or error rates
Pricing
There is a free version and a PRO version that costs $199/month or $1990/year
What Makes it Unique?
Easy to install
Good for runnign quick tests
Simple creation of YAML files whit less blocks.
Handles almost 1.500 requests/second on a average CPU
BlazeMeter is a vendor that provides performance and load testing as a service, and it now offers a brand new, complete Continuous Testing Platform to help enterprises shift-left. BlazeMeter provides a web-based interface for creating static load tests and can run any JMeter script for dynamic load tests. One of BlazeMeter’s main strengths is that it supplements JMeter (the top open source load testing app) with “Enterprise” features such as distributed testing, real time reporting, Integration with developer tools for continuous integration (CI) and application performance monitoring (APM).
Key Features
100% Compatible with Apache JMeter
Create tests on any scale – up to 1 million concurrent users
Set up tests within minutes
Run scriptless tests or upload URL lists
Run from the cloud or on-premise
Run tests from multiple geo-locations
Simulate mobile testing from real devices
Facilitates team collaboration with script and report sharing
Multi-tool support
Integrations with leading CI and APM tools
Support and professional services
Real-time, detailed reporting of load test results
Set KPIs for tested app behavior
Desktop and mobile user experience monitoring
Pricing
Free (50 concurrent users)
Basic: $99/mo (1000 concurrent users)*
Pro: $499/mo (5000 concurrent users)*
Unleashed: Contact vendor for pricing*
*Annual Pricing
What Makes it Unique?
BlazeMeter allows you to see test results in real time or easily compare multiple tests, all with a simple to use interface and intuitive graphs
It has a flexible pricing that accommodates hourly, monthly or annual usage. No setup or maintenance fees
BlazeMeter’s professional services team can help you to set up and run your performance tests
7. Dynatrace
Dynatrace is a widely used application performance management tool that comes with advanced features for monitoring Java and .NET applications, which aids to easily identify bottlenecks or errors in your application. PurePath technology used in Dynatrace provides end-to-end transaction level details from the browser, across all tiers and database. It helps to uncover performance issues even at the code level and also details of transactions invoking external services. This tool detects abnormalities in response time, transaction rate, throughput and system usage.
Key Features
Full stack discovery
Full stack performance management
Visualizing application infrastructure
AI-powered analytics
Self-healing at scale
Detecting vMotion events
Digital experience management
Container, cloud, and infrastructure management
Pricing
Dynatrace offers a free trial, contact the vendor for pricing
What Makes it Unique?
Dynatrace provides big data analytics
AI-optimized to find answers and actionable insights for DevOps teams
Can conduct automated monitoring of up to more than 100,000 hosts
8. Flood
Flood by Tricentis is a load testing platform that lets you run globally-distributed performance tests with your favorite open source tools, including JMeter, Gatling, and Selenium. Flood load tests can be scaled out for maximum concurrency and throughput at any given time. They will take care of the infrastructure and provide aggregated, real-time reporting.
Key Features
Simulate thousands of users in an autonomous fashion and scale out on demand to hundreds of thousands of users
Execute simple URL tests or more advanced test plans which are 100% compatible with JMeter or Gatling
Powerful and simple ruby-jmeter DSL
Build your own Grid: Use Dynatrace Grid nodes on demand or host your own
Report accurately with descriptive statistics at individual transaction level with raw data available in JSON or CSV formats
Only pay for the test load infrastructure you use, not per test or per simulated user
Share results from your own domain and brand it as your own with custom colour and logo configuration
Integrate test results with other cloud services such as New Relic and Flowdock
Pricing
Team: 50 node hours $299/mo
Enterprise: Contact the vendor for pricing information
What Makes it Unique?
Tool agnostic, can integrate with new tools as they become available
Cluster-less, shared-nothing architecture which means you can scale out for very large tests without the typical bottlenecks found in more traditional setups
Real time reports in an easy to share, intuitive, and interactive dashboard
Long term storage of results, so you can continuously assess performance over time
9. Gatling Frontline
Gatling is an open-source load and performance testing framework based on Scala, Akka and Netty. The software is designed to be used as a load testing tool for analyzing and measuring the performance of a variety of services, with a focus on web applications. Designed to facilitate continuous testing, it integrates with your build tool and offers a web recorder and colorful reports. Gatling also comes in a paid, enterprise version, Gatling FrontLine, which provides advanced analytics and integrations.
Key Features
Standalone HTTP Proxy Recorder
Scala-based scripting
An expressive self-explanatory DSL for test development
Asynchronous non-blocking engine for maximum performance
Excellent support of HTTP(S) protocols and can also be used for JDBC and JMS load testing
Validations and assertions
Comprehensive HTML Report
Pricing
Open Source version is free
Frontline On-premises: Unlimited testing, starting at 400€/mo
Frontline Cloud/based: Unlimited testing, starting at 3€/hour
What Makes it Unique?
Gatling is written in Scala, which allows you to run it on any system. This also means you won’t hit any trouble when using different local machines and cloud servers to run and create your tests. Further, scripts written in Scala are easy to maintain in a repository, Gatling is a good option to use in a CI/CD environment.
It has a metrics dashboard that you can see after test execution without having to add any additional plugins. The report is stored as an HTML file, which can be easily saved for later analyses.
It has an integrated assertions API, which gives you a full arsenal to perform functional assertions. This API allows you to run different types of functional checks along with your performance testing.
Tests are very elegant, because Gatling defines a domain-specific language that allows writing very clear and human readable tests.
Over the past years, we have gained a lot of experience with this tool, and we have a lot of further insights to share with you! Here are some blog posts that we have written that may help you with Gatling:
Load Impact is a cloud service to load test the performance of websites, apps and APIs – providing the tools business needs to be confident of their application’s performance. Its use cases include local testing, regression testing, cloud testing, and A/B testing. To use it, all you have to do is code your test in ES6 Javascript, or use the Load Impact GUI to generate the JavaScript necessary for your test. Convert existing Postman collections and JMeter tests. Then use the CLI to run tests locally from your machine or in its cloud, distributed across multiple geo locations, and analyze the results.
Key Features
Large scale load tests up to 1.2 million concurrent users
Accessible on-demand – no licensing or software to install
Supports any HTTP-based application or service
Mobile testing (client and network emulation)
Multi-geographic distribution of load
API testing
Automated and advanced scripting
Server monitoring
Easy-to-use scenario recorders
Browser emulation
Possibility to add extra IPs
Plugins for Jenkins & TeamCity
Test archive
REST API
Java and Python SDKs
Real-time data and dashboard
Professional services and support
Schedule tests (daily, weekly, monthly)
Interactive graphs
Load script APIs (e.g. JSON and XML parsing, etc.
Pricing
Free trial
Varying plans depending on cloud or local execution
What Makes it Unique?
Realistic load tests: Load Impact is capable of running load tests that mirror reality
Global load: In a single test, Load Impact can generate load from up to 10 different geographical locations from all over the world simultaneously
Heavy measuring: Load Impact does the heavy lifting in terms of calculating the response rate of every transferred resource. At this level, clients can gain high quality insight that allows them to determine and resolve issues immediately. Load Impact also measures CPU usage, Memory usage, Disk I/O and Network I/O.
11. LoadUI Pro
LoadUI is a load testing software, targeted mainly at web services that runs on Windows, Linux and Mac OS. It allows users to test the speed and scalability of APIs, preview API performance behaviors before releasing to production environments and shift performance insights to the left. LoadUI Pro is by SoapUI, as such, LoadUI Pro enables you to reuse your soapUI tests as load tests with just a click.
Key Features
Cloud-based API load tests: Prove your API can handle the big time through load testing from the cloud, using Amazon
Reuse existing functional tests: Use existing SoapUI Pro tests under different load test scenarios without reworking the original tests
For comprehensive API load testing, you need to employ multiple performance testing strategies. LoadUI Pro provides pre-built load strategies that you can configure to meet your needs
Parallel API load testing: Run several load test scenarios simultaneously to see how various conditions interact with each other and impact your API’s performance
Isolated load testing: Use LoadUI Pro with ServiceV Pro and/or VirtServerto run complex load scenarios against your virtualized assets so your load test results are not impacted by third-party APIs
Server monitoring to help you diagnose what resources are causing latency and bottlenecks
Pricing
LoadUI Pro Small – Fixed license: $4,999/year
LoadUI Pro Medium – Fixed license: $9,999/year
ReadyAPI – Fixed or floating license: Contact vendor
What Makes it Unique?
Simulate various types of load with 6 built-in strategies—the strategies included in LoadUI Pro are: Fixed, Burst, Ramp up, Ramp sequence, Custom, Variance
Enables automatic generation of data, capturing data from a database or a CSV and links these to your tests
Built to be compatible with SoapUI tests
12. NeoLoad
NeoLoad is a powerful load and performance testing software solution designed for web and mobile applications, which realistically simulates large numbers of users and analyzes your server behavior. Thanks to its monitoring modules, NeoLoad identifies performance bottlenecks, enabling you to correct them before they become expensive issues in production. It helps you to confidently deploy high-performance internet, intranet or mobile applications regardless of what technologies you may use.
Key Features
Broad technology support
Scriptless design and visual programming
Live debugging and monitoring
High load support
Network virtualization
Device/Browser Simulator
Cloud platform
Broad infrastructure monitoring support
Detailed reports and in-depth results analysis
It’s easy to collaborate as you can share test results, designs, and a pool of virtual users
Offers several integrations and APIs
Pricing
Free: For developers and testers running small load or performance tests
Standard: For one tester testing one application at a time*
Professional: For teams testing one application at a time*
Enterprise: For organizations testing multiple applications concurrently*
VUH: For organizations needing test size and licensing flexibility*
*Contact the vendor for pricing information
What Makes it Unique?
Supports performance testing of all standard and rich applications (RIA) for desktop and mobile devices
Monitors all major servers available on the market
Low cost licensing and maintenance, flexible licensing options
13. New Relic
New Relic is a web application performance service designed to work in real time with your live web app. New Relic Infrastructure provides flexible, dynamic server monitoring. Infrastructure empowers modern operation teams to make intelligent decisions about complex systems, from a physical datacenter to thousands of Amazon Elastic Compute Cloud (Amazon EC2) or Microsoft Azure instances. You can see the important performance data of your app in New Relic, like browser response time by geography and browser type, web transactions in real time, etc.
Key Features
Easy-to-set-up real-time instrumentation and analytics
Flexible instrumentation and dashboarding
Correlates application performance to end-user experience
Connects application and infrastructure performance
Rich, detailed transaction data
Real-time error analysis with on-demand diagnostic tools
Integration with DevOps tooling
Cloud-service instrumentation
Built to scale
Pricing
Price is based on the size of your instance, the number of instances, and the total time running. You can estimate your price here.
What Makes it Unique?
Works with all the different web development languages, making compatibility a non-issue
Works as a service so that you can access it from anywhere and anytime
Ready-to-go performance metrics dashboard allows you to drill down on the “browser page load time” to the amount of time the web application, network, DOM processing, page rendering, etc., taken at various instances of time
14. OctoPerf
OctoPerf is the first, full web user experience in the load testing tool market. Both their SaaS load testing solution and their on-premise Enterprise Edition comes with a Web UI. You can design, set your load policy, execute your load test, and analyze it directly from a web browser. OctoPerf is developed by a hard-core, bootstrapping agile team that places immense focus on user support with contextual documentation, video tutorials, a methodology training and a live chat.
Key Features
Full Apache JMeter support
Auto-correlations and frameworks
Codeless scripting engine
Cloud and on-premise load injection
Free and unlimited server monitoring engine
Live, customizable, and exportable analysis reports
On demand: As many concurrent users as you need, as many tests as you need (Contact the vendor for pricing information)
*Annual Pricing
What Makes it Unique?
Allows you to get deep insights about what’s happening on the server side: Monitor specific network interfaces, databases and even OS processes using OctoPerf’s monitoring or APM integrations
Compared to other SaaS load testing solutions, OctoPerf brings you the comfort of a fully integrated solution. From design to analysis, you can do everything in their web interface.
Allows you to easily create realistic tests thanks to their graphical scripting engine. Handle dynamic parameters in only a few clicks. Import any JMeter JMX file or Http Archive.
15. Redline 13
RedLine13 is focused on bringing (almost) free continuous load testing to the cloud. It provides a command and control architecture for storing, planning, starting, monitoring, and reporting performance tests at scale by using Amazon Web Services (AWS) cloud technology.
Key Features
Build packaged, custom, and repeatable tests that can easily be delivered to QA, streamlining your testing process.
Take complete control of your tests by running them in your own AWS environment. The benefits of cloud, with the controls of on-premise.
All RedLine13 user’s load tests are backed up and saved for your convenience. When you need to run the same tests again, simply hit the “clone test” button.
Run an unlimited amount of tests on an unlimited amount of servers for as long as you’d like.
Scale out your tests from 1,000 users even up to 1,000,000 users.
Access our own open-source load testing plugins, craft your own, or use one from UBIK to streamline your testing process
Integrates with Jenkins to complete the continuous integration process for load testing
From real-time graphs to generating downloadable reports, constantly monitor your load tests with its reporting features
Pricing
Free plan
Basic plan for single user: $75/mo; $500/year
Premium plan for a testing team: $299/mo; $2,500/year
Custom enterprise plan: Contact the vendor for pricing information
What Makes it Unique?
Open architecture for building and running load tests
Allows you to build packaged, custom, and repeatable tests
Allows you to take complete control of your tests by running them in your own AWS environment. The benefits of cloud, with the controls of on-premise.
16. Taurus
Created by BlazeMeter, Taurusis an open-source framework that provides a simple way to create and run performance tests, as well as an easy integration with additional open-source functional and performance testing software, like Selenium, Gatling or JMeter. Taurus uses YAML files, which are very easy to understand.
Key Features
An easily-readable, version control friendly and unified DSL (Domain-specific language) to define load test scenarios
Execute existing JMeter (or Grinder or Gatling or Selenium) tests
Create new tests from scratch using the aforementioned DSL
Merge multiple existing test scripts and/or DSL-driven tests into a single scenario.
Real-time reporting
JUnit-compatible test results format.
Easy way to define flexible pass/fail criteria. In case the results exceed a threshold, it is possible to automatically mark test(s) as failed.
Platform-independency: runs where Python and Java can run.
Apt tool to use in CI/CD environments
Pricing
Free / Open Source
What Makes it Unique?
Simple setup and upgrading
Integration with the BlazeMeter reporting service
Console stats and pseudo-graphic charts during the test execution (which can be combined with the BlazeMeter integration
And there you have it, some of our most recommended tools for performance testing and monitoring! They all have their own advantages for performance engineering.
Is there a tool you recommend that we didn’t cover here? Let us know!
Looking at some of the challenges of test automaton and how to pick the best tools and infrastructure In today’s Quality Sense episode, Federico Toledo sits down for a chat with the founder and CEO of Testim, Oren Rubin, an Israeli entrepreneur who has over…
WOPR29 is coming and we wonder: how does the current and global situation affects performance testing services? And how does it help to overcome historical moments? Let’s find out in this article, with the opinions of Matías Reina, Roger Abelenda, and Paz de Soto, as…
Hi, Thanks for sharing such wonderful tools!! I want to share one of the best Performance Testing tools to emphasize software testing. ClicLoad is packed with easy-to-use features for an end to end testing and it stimulates the load realistically to glean the knowledge of your application performance to your customer. https://www.clictest.com/products/clicload/
Hi, Thanks for sharing such wonderful tools!! I want to share one of the best Performance Testing tools to emphasize software testing. ClicLoad is packed with easy-to-use features for an end to end testing and it stimulates the load realistically to glean the knowledge of your application performance to your customer.
https://www.clictest.com/products/clicload/
Got a clear idea about the tools and its features for performance testing. Thanks for sharing such an informative article.