Key considerations and strategies for going the open source route

If you’re paying for expensive software testing tool licences, perhaps something to consider is migrating to open source alternatives to optimize costs, especially if your team needs to find ways to go lean during the pandemic. 

Of course, like most things worthwhile, migrating implies some up-front investment, with the realization of cost savings happening in the mid to long-term. 

Open source tools are known for their flexibility and several of them have reached world-class quality with a gigantic community around them (think JMeter, Cucumber, Selenium, Appium…), making finding the support you need a breeze. 

We’ll admit, there are cases when it may not be that convenient to migrate to open source. For example, a team that’s already using JIRA and has it integrated with many other tools probably won’t opt to migrate from JIRA to Mantis Bug Tracker. But, there are many other cases when it does make a lot of sense. What I want to encourage you in this post, is to analyze the ROI related to the different tools you are using (and paying licenses for), considering the current context of your team and company, and think about what’s the best option and how costly would it be to migrate if that’s the best one. 

To set an initial example, if today you use LoadRunner for performance tests, it’s quite simple to understand and adopt JMeter or Gatling instead, which in my view, are even better tools most of the time. 

QUICK TIP: In case you want to migrate your existing test suite from LoadRunner to JMeter, it can be done automatically with BlazeMeter’s converter.

In this post, we’ll go over what to take into account when choosing an open source tool and then explain some strategies for making your migration a successful one.  

Choosing an Open Source Testing Tool

There are some factors that, in my opinion, are very important when choosing which open source tool to migrate to, since, there are generally several alternatives available for what you need. 

So how do you decide?

Community

An open source tool’s community says quite a lot about it. As Refael said in my podcast interview, “if the open source product is successful, there is a community behind it.” Of course, the bigger, the better! See if there’s a sizable enough community from which you could get support, advice, browse forum discussions, etc. This is also important in order to know how easy it will be to find people who’re skilled in the particular when looking to recruit. Also, check to see if there’s a decent amount of training available, as this is another sign that you can find people who know how to use it or train your own people to use it. 

How often is it updated? 

Check out the tool’s repository in GitHub or where its code is hosted. For example, if you wanted to learn more about Taurus for performance testing, you could scope it out here: https://github.com/Blazemeter/taurus.

In its repository, you can see the amount of contributors, when the last update was, its list of open issues, when the last release was and how often they happen.

Background technology

It’s important to think for the future. If the tool is built with Visual Basic, it probably won’t have much of a future, no matter how many contributors it has or if they released a new version recently.

These recommendations are just a start for when it comes to what you should consider about open source tools. Of course, you’ll have to drill down to the more nitty-gritty as well, given the type of tool and your needs. For example, think about if it would be compatible with your technology, skills, intended usage, necessary integrations, performance, etc.

Strategies for Migrating to Open Source Tools

Once you decide on which tool, platform or set of tools you will migrate to, the next question is: How can you pull off the migration without losing the benefits of what your current tool has provided you?

Here are some ideas:

• First, find out if someone else has done the same migration before and was successful.
• When doing this research, see if there are any tools or scripts that may help by doing some of the heavy lifting for you. As we mentioned before, for the Loadrunner to JMeter example, we’ve been working with BlazeMeter on a converter just for this purpose. You can check it out here.
• Another possibility is to find out if there are any companies that could facilitate this migration for you (of course I have to mention that, as we in Abstracta have quite a lot of experience doing so 🙂).
• Last but not least, follow a risk-based approach or ROI-based approach. This means analyze which are the test cases that are worth migrating first.

3 Ways to Migrate

For the migration itself, the strategies we suggest are the following (let’s assume we’re migrating from one automation tool to another, but these are applicable for other scenarios too):

Big Bang Migration

Migrate all at once, work on the migration and do not substitute one tool for the other until you have everything done. 

Progressive migration

Migrate case by case and continue executing with both tools, but with the new one, run the migrated cases and with the old one, run those that have yet to be migrated. 

Migrate in Parallel

Run two executions in parallel with all the cases in the old tool and the already migrated cases in the new tool throughout the whole migration. Analyze the differences in the test results. This is the best strategy in our opinion since you can test the new tool against the old, gain confidence, and make sure the results in both solutions are the same. There is one possible problem though with this strategy, which is the availability of the test environment. Do you  have enough time to execute the tests twice?

So what about you? Have you ever done any testing tool migrations? What’s your experience with open source testing tools?

Recommended for You

How Can You Optimize the Cost of Software Testing?
Shift-Left Testing and the Case for Open Source in the Enterprise