With all of the free and enterprise tools available for performance testing, there’s no excuse for having a system failure

MAY 2020 UPDATE – We’ve added some more tools to our original list: Apptim and Artillery 🙂

Performance tests reveal how a system behaves and responds during various situations. A system may run very well with only 1,000 concurrent users, but how would it run with 100,000? In terms of performance, we wish to achieve three main goals: speed, scalability, and stability.

So, to understand how your site or app behaves when 100 people access it at once, do you call all of your friends and have them all enter at the exact same time? That could be one way to run a load test… Or, you can use a tool specifically designed to simulate that kind of load and much more! 

Here, we’ve made a list of our own favorite performance/load testing and application performance monitoring tools to help you build and maintain a highly reliable and performant system (In an unbiased, alphabetical order!). Several of these are open source and most of the paid ones offer free trials. 

1. AgileLoad

AgileLoad is an enterprise-class performance testing solution for optimizing the performance of your web applications. AgileLoad promises to increase your ROI and productivity, allowing you to perform your tests faster while providing a deep analysis of test results and full support for all new technologies. It successfully gives in-depth performance diagnostics for a wide range of applications from those using the most advanced web2.0 technologies to more classic client/server or heavy clients.

Key Features

• Intuitive graphic interface lets you capture and design sophisticated business scenarios
• Automatic detection and analysis of performance issues
• Highly customizable scripting and reporting engine 
• Support performance testing across the application lifecycle
• Test against a broad range of applications and protocols
• Record and replay RIAs using a variety of Web 2.0 technologies
• Measurable and repeatable load generation with system and end user monitoring
• Automatic recording of test scenarios
• Simulate realistic user load with Agileload load injection engine
• Runs under Microsoft Windows (2003 and higher) and is compatible with all web browsers

Pricing

• Free version: Build all your test scenarios with 10 virtual users
• Pay as you go: Power up your Agileload by renting from 50 to 10000 virtual users

What Makes it Unique?

• The free version or “starter kit” allows for quick ramp-up use of the tool
• Automatic modeling vastly reduces scripting time for subsequent test cases against the same app
• End-to-end diagnostic features: The tool not only monitors the front-end, but also the back-end to detect the reason for performance degradations
• Anomaly detection and suggestion features help quickly identify and resolve performance bottlenecks
• Configurable reports, for a mix of audiences

2. Apache JMeter

The de facto standard and “king” of load testing tools, Apache JMeter is an open source, 100% java based application with a graphical interface. It is designed to analyze and measure the performance and load functional behavior of web applications and a variety of services.

JMeter allows you to perform various testing activities like performance, load, stress, regression and functional testing, in order to get accurate performance metrics against your web server. The tool supports variable parameterization, assertions (response validation), per-thread cookies, configuration variables and a variety of reports. JMeter users heavily rely on its various plugins as most of its “out of the box” features are implemented through them.

Key Features

• User-friendly GUI: JMeter comes with a simple and interactive GUI
• Platform independent: JMeter is written and developed using java, so it can run on any environment or workstation that accepts a Java virtual machine, for example – Windows, Linux, Mac, etc.
• Supports various server types: Web (HTTP, HTTPS, SOAP), database (JDBC, LDAP, JMS) and mail (POP3)
• Supports multiple protocols such as HTTP, JDBC, LDAP, SOAP, JMS, and FTP
• Simulates multiple users by using virtual users or unique users in order to generate heavy load against web application under test
• Multi-threading framework allows concurrent and simultaneous sampling of different functions by many or separate thread groups
• Remote distributed testing: JMeter uses a Master-Slave concept for distributed testing where the master will distribute tests among all the slaves and the slaves will execute scripts against your server
• Test results can be viewed in different formats like graph, table, tree, and report etc.

Pricing

Free / Open Source

What Makes it Unique?

• JMeter has a very large community around it with endless resources and integrates with almost every performance testing tool on the market
• Easy installation: It can be installed on any desktop with Windows, Mac or Linux
• The test IDE allows test recording from browsers or native applications
• Readily available plugins, for example, visualization plugin for data analysis

3. Appdynamics 

AppDynamics is a leading tool that helps analyze, optimize and predict bottlenecks in complex information systems. It can be done on site, during product test and deployment and in the cloud. It has agents for .Net, node js, PHP, Java, C++, Python, Apache Web Server and GoLang. It allows not only real-time application performance monitoring, but business performance monitoring and end user monitoring. According to AppDynamics, its platform helps you to connect application performance and customer experience to your business outcomes with a smarter approach to performance management.

Key Features

• Code level visibility
• Dynamic baselining and powerful alerting
• Application performance management agents
• Data retention
• Monitoring of multiple platforms
• Troubleshoot performance issues in production
• Trend database performance over time
• Continuously monitor in a high-volume environment
• Install and begin using immediately
• Database agents
• Server Visibility
• Synthetic Monitoring
• Mobile Real-User Monitoring
• Browser Real-User Monitoring

Pricing

There are 3 tiers: APM Pro, APM Advanced, and APM Peak – contact the vendor for pricing

What Makes it Unique?

• Ensures that businesses get useful information with its unified analytics functionality.
• AppDynamics features an innovative approach called Unified Monitoring that incorporates various AppDynamics applications into a single robust solution that helps to determine the precise origin of any performance issue

4. Apptim

Apptim was co-founded by an Abstracta co-founder, Fabian Baptista, and Sofia Palamarchuk, our former CEO. The vision behind it is to make autonomous mobile testing a reality. The tool has been in beta until now. Today it offers an easy way for developers and testers to quickly gain an understanding of their mobile app’s performance, usability, and functionality.

Key Features

• Test your mobile app on real devices
• Supports manual and exploratory testing
• Mobile performance dashboard with key metrics and rich data around resource usage, power consumption, start times, crashes, etc.
• Integrates with Jira
• iOS and Android testing
• Performance troubleshooting and tuning

Pricing

Apptim is free – enterprise solutions available upon request.

What Makes it Unique?

This tool makes it effortless to fully test your mobile app in every build. Its biggest strength lies in its ability to help you to easily troubleshoot your app for crashes and exceptions while giving you full visibility on app performance trends.

5. Artillery

Artillery is a load testing and functional testing toolkit written in Node.js. It’s main purpose is to simulate realistic load on complex applications, it works with the concepts of virtual users that arrive to use the application in phases. You write your load testing scripts in YAML, with the option to use Javascript to write custom testing logic.

Key Features

• Works from the command line
• Easy integration with additional software for working with continuous integration
• Less resources consumption while running
• Works with HTTP(S), Socket.io, WebSockets, and AWS Kinesis
• Simple JSON reports that can be exported and shared
• Lets you create various scenarios and add them a weight property, the scenarios with higher weight have higher chances to be picked by the virtual users
• Can set success conditions for latency or error rates

Pricing

There is a free version and a PRO version that costs $199/month or $1990/year

What Makes it Unique?

• Easy to install
• Good for runnign quick tests
• Simple creation of YAML files whit less blocks.
• Handles almost 1.500 requests/second on a average CPU

For more information about Artilley, see its documentation page: https://artillery.io/docs/

6. BlazeMeter

BlazeMeter is a vendor that provides performance and load testing as a service, and it now offers a brand new, complete Continuous Testing Platform to help enterprises shift-left. BlazeMeter provides a web-based interface for creating static load tests and can run any JMeter script for dynamic load tests. One of BlazeMeter’s main strengths is that it supplements JMeter (the top open source load testing app) with “Enterprise” features such as distributed testing, real time reporting, Integration with developer tools for continuous integration (CI) and application performance monitoring (APM).

Key Features

• 100% Compatible with Apache JMeter
• Create tests on any scale – up to 1 million concurrent users
• Set up tests within minutes
• Run scriptless tests or upload URL lists
• Run from the cloud or on-premise
• Run tests from multiple geo-locations
• Simulate mobile testing from real devices
• Facilitates team collaboration with script and report sharing
• Multi-tool support
• Integrations with leading CI and APM tools
• Support and professional services
• Real-time, detailed reporting of load test results
• Set KPIs for tested app behavior
• Desktop and mobile user experience monitoring

Pricing

• Free (50 concurrent users)
• Basic: $99/mo (1000 concurrent users)*
• Pro: $499/mo (5000 concurrent users)*
• Unleashed: Contact vendor for pricing*

*Annual Pricing

What Makes it Unique?

• BlazeMeter allows you to see test results in real time or easily compare multiple tests, all with a simple to use interface and intuitive graphs
• It has a flexible pricing that accommodates hourly, monthly or annual usage. No setup or maintenance fees
• BlazeMeter’s professional services team can help you to set up and run your performance tests

7. Dynatrace

Dynatrace is a widely used application performance management tool that comes with advanced features for monitoring Java and .NET applications, which aids to easily identify bottlenecks or errors in your application. PurePath technology used in Dynatrace provides end-to-end transaction level details from the browser, across all tiers and database. It helps to uncover performance issues even at the code level and also details of transactions invoking external services. This tool detects abnormalities in response time, transaction rate, throughput and system usage.

Key Features

• Full stack discovery
• Full stack performance management
• Visualizing application infrastructure
• AI-powered analytics
• Self-healing at scale
• Detecting vMotion events
• Digital experience management
• Container, cloud, and infrastructure management

Pricing

Dynatrace offers a free trial, contact the vendor for pricing

What Makes it Unique?

• Dynatrace provides big data analytics
• AI-optimized to find answers and actionable insights for DevOps teams
• Can conduct automated monitoring of up to more than 100,000 hosts

8. Flood

Flood by Tricentis is a load testing platform that lets you run globally-distributed performance tests with your favorite open  source tools, including JMeter, Gatling, and Selenium. Flood load tests can be scaled out for maximum concurrency and throughput at any given time. They will take care of the infrastructure and provide aggregated, real-time reporting.

Key Features

• Simulate thousands of users in an autonomous fashion and scale out on demand to hundreds of thousands of users
• Execute simple URL tests or more advanced test plans which are 100% compatible with JMeter or Gatling
• Powerful and simple ruby-jmeter DSL 
• Build your own Grid: Use Dynatrace Grid nodes on demand or host your own
• Report accurately with descriptive statistics at individual transaction level with raw data available in JSON or CSV formats
• Only pay for the test load infrastructure you use, not per test or per simulated user
• Share results from your own domain and brand it as your own with custom colour and logo configuration
• Integrate test results with other cloud services such as New Relic and Flowdock

Pricing

• Team: 50 node hours $299/mo
• Enterprise: Contact the vendor for pricing information

What Makes it Unique?

• Tool agnostic, can integrate with new tools as they become available
• Cluster-less, shared-nothing architecture which means you can scale out for very large tests without the typical bottlenecks found in more traditional setups
• Real time reports in an easy to share, intuitive, and interactive dashboard
• Long term storage of results, so you can continuously assess performance over time

9. Gatling Frontline

Gatling is an open-source load and performance testing framework based on Scala, Akka and Netty. The software is designed to be used as a load testing tool for analyzing and measuring the performance of a variety of services, with a focus on web applications. Designed to facilitate continuous testing, it integrates with your build tool and offers a web recorder and colorful reports. Gatling also comes in a paid, enterprise version, Gatling FrontLine, which provides advanced analytics and integrations. 

Key Features

• Standalone HTTP Proxy Recorder
• Scala-based scripting
• An expressive self-explanatory DSL for test development
• Asynchronous non-blocking engine for maximum performance
• Excellent support of HTTP(S) protocols and can also be used for JDBC and JMS load testing
• Validations and assertions
• Comprehensive HTML Report

Pricing

• Open Source version is free
• Frontline On-premises: Unlimited testing, starting at  400€/mo
• Frontline Cloud/based: Unlimited testing, starting at 3€/hour

What Makes it Unique?

• Gatling is written in Scala, which allows you to run it on any system. This also means you won’t hit any trouble when using different local machines and cloud servers to run and create your tests. Further, scripts written in Scala are easy to maintain in a repository, Gatling is a good option to use in a CI/CD environment.
• It has a metrics dashboard that you can see after test execution without having to add any additional plugins. The report is stored as an HTML file, which can be easily saved for later analyses.
• It has an integrated assertions API, which gives you a full arsenal to perform functional assertions. This API allows you to run different types of functional checks along with your performance testing.
• Tests are very elegant, because Gatling defines a domain-specific language that allows writing very clear and human readable tests. 

Over the past years, we have gained a lot of experience with this tool, and we have a lot of further insights to share with you! Here are some blog posts that we have written that may help you with Gatling: 

Gatling Tool Review

Gatling vs. JMeter: Our Findings

Gatling Performance Testing Pros and Cons

How to Implement Load Test Scenarios in Gatling

A Tutorial on How to Use Gatling with Maven

10. Load Impact

Load Impact is a cloud service to load test the performance of websites, apps and APIs – providing the tools business needs to be confident of their application’s performance. Its use cases include local testing, regression testing, cloud testing, and A/B testing. To use it, all you have to do is code your test in ES6 Javascript, or use the Load Impact GUI to generate the JavaScript necessary for your test. Convert existing Postman collections and JMeter tests. Then use the CLI to run tests locally from your machine or in its cloud, distributed across multiple geo locations, and analyze the results.

Key Features

• Large scale load tests up to 1.2 million concurrent users
• Accessible on-demand – no licensing or software to install
• Supports any HTTP-based application or service
• Mobile testing (client and network emulation)
• Multi-geographic distribution of load
• API testing
• Automated and advanced scripting
• Server monitoring
• Easy-to-use scenario recorders
• Browser emulation
• Possibility to add extra IPs
• Plugins for Jenkins & TeamCity
• Test archive
• REST API
• Java and Python SDKs
• Real-time data and dashboard
• Professional services and support
• Schedule tests (daily, weekly, monthly)
• Interactive graphs
• Load script APIs (e.g. JSON and XML parsing, etc.

Pricing

• Free trial
• Varying plans depending on cloud or local execution

What Makes it Unique?

• Realistic load tests: Load Impact is capable of running load tests that mirror reality 
• Global load: In a single test, Load Impact can generate load from up to 10 different geographical locations from all over the world simultaneously
• Heavy measuring: Load Impact does the heavy lifting in terms of calculating the response rate of every transferred resource. At this level, clients can gain high quality insight that allows them to determine and resolve issues immediately. Load Impact also measures CPU usage, Memory usage, Disk I/O and Network I/O.

11. LoadUI Pro

LoadUI is a load testing software, targeted mainly at web services that runs on Windows, Linux and Mac OS. It allows users to test the speed and scalability of APIs, preview API performance behaviors before releasing to production environments and shift performance insights to the left. LoadUI Pro is by SoapUI, as such, LoadUI Pro enables you to reuse your soapUI tests as load tests with just a click. 

Key Features

• Cloud-based API load tests: Prove your API can handle the big time through load testing from the cloud, using Amazon
• Reuse existing functional tests: Use existing SoapUI Pro tests under different load test scenarios without reworking the original tests
• For comprehensive API load testing, you need to employ multiple performance testing strategies. LoadUI Pro provides pre-built load strategies that you can configure to meet your needs
• Parallel API load testing: Run several load test scenarios simultaneously to see how various conditions interact with each other and impact your API’s performance
• Isolated load testing: Use LoadUI Pro with ServiceV Pro and/or VirtServerto run complex load scenarios against your virtualized assets so your load test results are not impacted by third-party APIs
• Server monitoring to help you diagnose what resources are causing latency and bottlenecks

Pricing

• LoadUI Pro Small – Fixed license: $4,999/year
• LoadUI Pro Medium – Fixed license: $9,999/year
• ReadyAPI – Fixed or floating license: Contact vendor

What Makes it Unique?

• Simulate various types of load with 6 built-in strategies—the strategies included in LoadUI Pro are: Fixed, Burst, Ramp up, Ramp sequence, Custom, Variance
• Enables automatic generation of data, capturing data from a database or a CSV and links these to your tests
• Built to be compatible with SoapUI tests

12. NeoLoad

NeoLoad is a powerful load and performance testing software solution designed for web and mobile applications, which realistically simulates large numbers of users and analyzes your server behavior. Thanks to its monitoring modules, NeoLoad identifies performance bottlenecks, enabling you to correct them before they become expensive issues in production. It helps you to confidently deploy high-performance internet, intranet or mobile applications regardless of what technologies you may use.

Key Features

• Broad technology support
• Scriptless design and visual programming
• Live debugging and monitoring
• High load support
• Network virtualization
• Device/Browser Simulator
• Cloud platform
• Broad infrastructure monitoring support 
• Detailed reports and in-depth results analysis
• It’s easy to collaborate as you can share test results, designs, and a pool of virtual users
• Offers several integrations and APIs

Pricing

• Free: For developers and testers running small load or performance tests
• Standard: For one tester testing one application at a time*
• Professional: For teams testing one application at a time*
• Enterprise: For organizations testing multiple applications concurrently*
• VUH: For organizations needing test size and licensing flexibility*

*Contact the vendor for pricing information

What Makes it Unique?

• Supports performance testing of all standard and rich applications (RIA) for desktop and mobile devices
• Monitors all major servers available on the market
• Enterprise-class: Efficient and infinitely scalable, infrastructure monitoring capability, APM tool integration
• Easy-to-use load testing tool: Scriptless test creation, GUI-driven design, automated parameterization
• Low cost licensing and maintenance, flexible licensing options

13. New Relic

New Relic is a web application performance service designed to work in real time with your live web app. New Relic Infrastructure provides flexible, dynamic server monitoring. Infrastructure empowers modern operation teams to make intelligent decisions about complex systems, from a physical datacenter to thousands of Amazon Elastic Compute Cloud (Amazon EC2) or Microsoft Azure instances. You can see the important performance data of your app in New Relic, like browser response time by geography and browser type, web transactions in real time, etc. 

Key Features

• Easy-to-set-up real-time instrumentation and analytics
• Flexible instrumentation and dashboarding
• Correlates application performance to end-user experience
• Connects application and infrastructure performance
• Rich, detailed transaction data
• Real-time error analysis with on-demand diagnostic tools
• Integration with DevOps tooling
• Cloud-service instrumentation
• Built to scale

Pricing

Price is based on the size of your instance, the number of instances, and the total time running. You can estimate your price here.

What Makes it Unique?

• Works with all the different web development languages, making compatibility a non-issue
• Works as a service so that you can access it from anywhere and anytime
• Ready-to-go performance metrics dashboard allows you to drill down on the “browser page load time” to the amount of time the web application, network, DOM processing, page rendering, etc., taken at various instances of time

14. OctoPerf

OctoPerf is the first, full web user experience in the load testing tool market. Both their SaaS load testing solution and their on-premise Enterprise Edition comes with a Web UI. You can design, set your load policy, execute your load test, and analyze it directly from a web browser. OctoPerf is developed by a hard-core, bootstrapping agile team that places immense focus on user support with contextual documentation, video tutorials, a methodology training and a live chat. 

Key Features

• Full Apache JMeter support
• Auto-correlations and frameworks
• Codeless scripting engine
• Cloud and on-premise load injection
• Free and unlimited server monitoring engine
• Live, customizable, and exportable analysis reports
• Results trends and comparison
• Collaborative work using Workspaces
• Fine grained Service Level Agreement
• APM integrations
• Jenkins integration and REST API

Pricing

• Free: 50 concurrent users/test, unlimited tests, 2 load generators
• Basic: 1,000 concurrent users/test, 200 tests/year, unlimited load generators 99€/mo*
• Pro: 3,000 concurrent users/test, 250 tests/year, unlimited load generator 499€/mo*
• On demand: As many concurrent users as you need, as many tests as you need (Contact the vendor for pricing information)

*Annual Pricing

What Makes it Unique?

• Allows you to get deep insights about what’s happening on the server side: Monitor specific network interfaces, databases and even OS processes using OctoPerf’s monitoring or APM integrations
• Compared to other SaaS load testing solutions, OctoPerf brings you the comfort of a fully integrated solution. From design to analysis, you can do everything in their web interface.
• Allows you to easily create realistic tests thanks to their graphical scripting engine. Handle dynamic parameters in only a few clicks. Import any JMeter JMX file or Http Archive.

15. Redline 13

RedLine13 is focused on bringing (almost) free continuous load testing to the cloud. It provides a command and control architecture for storing, planning, starting, monitoring, and reporting performance tests at scale by using Amazon Web Services (AWS) cloud technology.

Key Features

• Build packaged, custom, and repeatable tests that can easily be delivered to QA, streamlining your testing process.
• Take complete control of your tests by running them in your own AWS environment. The benefits of cloud, with the controls of on-premise.
• All RedLine13 user’s load tests are backed up and saved for your convenience. When you need to run the same tests again, simply hit the “clone test” button.
• Run an unlimited amount of tests on an unlimited amount of servers for as long as you’d like. 
• Scale out your tests from 1,000 users even up to 1,000,000 users.
• Access our own open-source load testing plugins, craft your own, or use one from UBIK to streamline your testing process
• Integrates with Jenkins to complete the continuous integration process for load testing
• From real-time graphs to generating downloadable reports, constantly monitor your load tests with its reporting features

Pricing

• Free plan
• Basic plan for single user: $75/mo; $500/year
• Premium plan for a testing team: $299/mo; $2,500/year
• Custom enterprise plan: Contact the vendor for pricing information

What Makes it Unique?

• Open architecture for building and running load tests
• Allows you to build packaged, custom, and repeatable tests 
• Allows you to take complete control of your tests by running them in your own AWS environment. The benefits of cloud, with the controls of on-premise.

16. Taurus

Created by BlazeMeter, Taurus is an open-source framework that provides a simple way to create and run performance tests, as well as an easy integration with additional open-source functional and performance testing software, like Selenium, Gatling or JMeter. Taurus uses YAML files, which are very easy to understand.

Key Features

• An easily-readable, version control friendly and unified DSL (Domain-specific language) to define load test scenarios
• Execute existing JMeter (or Grinder or Gatling or Selenium) tests
• Create new tests from scratch using the aforementioned DSL
• Merge multiple existing test scripts and/or DSL-driven tests into a single scenario.
• Real-time reporting
• JUnit-compatible test results format.
• Easy way to define flexible pass/fail criteria. In case the results exceed a threshold, it is possible to automatically mark test(s) as failed.
• Platform-independency: runs where Python and Java can run.
• Apt tool to use in CI/CD environments

Pricing

Free / Open Source

What Makes it Unique?

• Simple setup and upgrading
• Integration with the BlazeMeter reporting service
• Console stats and pseudo-graphic charts during the test execution (which can be combined with the BlazeMeter integration

For more information, read this Introduction to Taurus.

—

And there you have it, some of our most recommended tools for performance testing and monitoring! They all have their own advantages for performance engineering. 

Is there a tool you recommend that we didn’t cover here? Let us know!

Recommended for You

How to Make a Performance Test Plan
When is the Best Time to Start Performance Testing?