Ways to reduce the cost of software testing without sacrificing quality

Due to the novel coronavirus and the impact it’s having on the global economy, my colleagues and I have been thinking a lot these days about how we could help our clients and others find ways to reduce the cost of testing without letting their products’ quality suffer. And dare we say it, maybe even lessen the need of having to let go of valuable team members. In certain cases and many industries, the latter is unavoidable but it’s important to first try to control or optimize costs as much as possible. 

In this post, we’ll share some ways to reduce software testing costs that we have tried and tested firsthand at Abstracta with great results.

Optimize Testing Costs

Test as Soon as Possible

Just like it’s important to test for early detection of an illness, it’s important to test your software early, before small issues (bugs) start to develop into much larger (and expensive) ones. By applying the shift-left testing approach, testing becomes more efficient because a tester’s learning curve can be tracked to the process. Asking questions sooner, like “What would happen if…” will help to improve the requirements and reduce the amount of re-work.

Recommended Reading: Not Convinced Yet About Shift-Left Testing?

Be Careful with Documentation

Be careful with how much tests are documented (the requirements and other things as well), look for a good balance. More documentation helps new people learn faster, helps to clarify, to discuss things in black and white. On the other hand, it makes the process less flexible, adding maintenance costs to keep that documentation up to date with the continuously changing requirements. Aim to find the right balance. 

Apply Risk-Based Testing for Everything

Looking at risk, we see it being made up of two factors: the probability of something happening and the level of (negative) business impact that it would have. Risk-based testing is all about prioritizing, prioritizing, prioritizing. Applying Pareto’s Principle, what are the 20% of test cases that would mitigate 80% of the risk? 

Recommended Reading: The Software Testing Risk Matrix

Use Production Information to Improve Testing

While it’s imperative to shift left testing, it’s also very important to shift right testing. The aim of this approach is to not let your users be the only ones testing your software in production!

One source of useful information for real user behavior is Google Analytics, for example, if you want to see what areas of your e-commerce website are most heavily trafficked (thus should be more rigorously tested). You could also check Google Analytics to understand from which devices, OS versions or browsers most of your users are accessing your site from so that you can simulate these behaviors in your testing.

Another example of obtaining production info is to investigate within your performance monitoring tool (for example, with your favorite APM tool) to understand how each set of changes in the code impacts performance. 

Optimize Automation Costs

Focus Automation at the API Level Rather than the UI Level 

Test automation, when applied wisely, can provide a great return on investment, despite the initial investment required.

You should decide at which level you will put more effort in automation, thinking of the different layers of your application. For us testers, we can automate at the UI level or at the API level. To understand more about the pros and cons about these options, check this article on the agile testing pyramid. 

One possible strategy is to start by automating the main flow of the functionality or the user story you want to test at the UI level, with the most common data. Then analyze which endpoints are invoked in that flow. Add automated tests at the API level for each endpoint and expand the coverage there with different test data, making interesting combinations. These tests are quicker to run, more robust (less maintenance), and run faster. Also, there are several free tools to do them like Postman and SoapUI.

Recommended Reading: When to Automate a Test? & Best Testing Practices for Agile Teams: the Automation Pyramid

Consider Migrating to Open Source Tools

If you’re paying for very expensive tool licenses, you could consider open source ones. Open source testing tools offer a high level of flexibility and have reached a level of maturity where there is much more confidence in them these days. 

With the significant expansion of the open source community, thousands of contributors are sharing their add-ons and solutions with the world, making everyone’s lives that much easier. 

For example, if you have LoadRunner for performance tests today, you can easily migrate to JMeter or Gatling which, from my point of view, are much better.

It’s important to note that in many cases, this does imply a large initial investment to migrate what you’ve got set up in one tool to another, but in the long term, it will generate cost savings. There are some other cases where the migration can be done automatically. For example, there is a converter by BlazeMeter that helps you to migrate from LoadRunner to JMeter.

Recommended Reading: Shift-Left Testing in the Enterprise and the Case for Open Source 

Optimize Infrastructure and Tooling Costs

Review Licenses and Subscriptions

In times of abundance, we’re not quite as concerned with investments, especially small ones. You see a tool, a membership, a subscription to a service or magazine and you just sign up, no questions asked because it’s under a certain threshold. When in crisis, it’s necessary need to review and cut out the inessentials. 

Tools are a big one here. Analyze the actual use being made of the tools you are paying for and perhaps adjust to a plan that better meets your needs. For example, check which features you routinely use and those which you don’t, seeing if there is a less expensive plan that covers most of the features you absolutely need.

Review the Use of Virtual Machines in the Cloud 

You may have cloud-based virtual machines (such as instances of Amazon, Azure, etc.) that you’re using that could actually be optimized in some way (having a single instance for two systems of little use, etc.), or turning off the ones you use only a few times per month. 

Optimize Training Costs

Encourage Internal Knowledge Sharing

At Abstracta, we’re very mindful about how we train our testers. For us, it’s a vital part of our operations, as we are constantly working to help our testers, especially juniors, grow (as senior testing roles can be hard to fill). We tend to hire more for potential and attitude since we’ve got training down pretty well. Many of us have experience teaching university courses and we’ve even launched our online learning platform, Abstracta Academy.

When it comes to expensive training packages and workshops from external providers, we generally sign up just for very specific topics and mostly for those of us in the company who possess senior roles. The reason for this is that our technical leaders and senior testers will have the best chance to get the most out of these trainings and they can then transmit what they learned to the others. 

Under the same idea, and following our belief that juniors have a lot to learn from the rest of the team, we aim for their training to come directly from our seniors. This helps us to reduce costs on outside training and also gives seniors the opportunity to develop important skills such as communication, leadership, public speaking, etc. 

Consider Online Training Platforms

As I mentioned, we went all out and made our own online platform, but, there are many software testing and development courses offered at affordable prices on Udemy, Codea, etc. In addition, there are excellent courses absolutely free, such as those from Test Automation University and Blazemeter University. 

Attend Webinars and Online Conferences

Same as above, the internet is your friend! Especially since conferences are being canceled, and so are in-person meetups, webinars and virtual conferences are becoming more prevalent and more often than not, free.

You can see our own webinars on our Youtube channel, or browse the Youtube channels of different testing conference organizers.

Recommended Reading: Best Software Testing Conferences in 2020 (Covid-19 Update) 

Optimize Process Costs

Last but not least, inefficiencies in processes lead to waste—a waste of time and money. So, here are some ways you could improve your processes:

• Analyze parts of the development process that generate waiting periods or bottlenecks, dependencies, unnecessary accumulation, etc. 
• Analyze communication channels and the problems that may be occurring (there are always communication problems) to find ways to improve and clarify, avoiding rework, errors, etc. 
• Take an agile approach. This implies avoiding the planning of large development cycles without getting feedback. Aim for short iterations, checking frequently with users or clients if expectations are being met. Analyze in each iteration how well the team has been working, aiming to make retrospective analyses that allow for future improvement.
• Conduct a root cause analysis of recurring errors, not only to work on solving these issues, but to understand the “why” behind them to avoid them in the future.

If Needed, Outsource Software Testing

Outsourcing is ideal for companies who need to focus their attention on their core competencies. If your team is not very mature in its test efforts or your software quality program and decisions are not delivering the desired outcomes, it may be wise to seek external help instead of prolonging the time spent using inefficient and ineffective processes.

When you do not have sufficient resources or skills in-house, it can be a better option than hiring someone directly due to the overhead costs savings and the outsourcing partner will be ready to go, with the ability to flexibly and seamlessly ramp up or down. Furthermore, added value can be extracted from outsourcing as you benefit from the collective expertise of the entire firm, not only that of the testers they dedicate to you.  

Lastly, outsourcing testing can help you uphold business continuity. When you hire someone in-house and they leave your company, or if they are the sole person in charge of testing and they take a vacation, you may be left scrambling. When you find someone else to fill the role, you have to enter the whole training process all over again. By outsourcing, the service provider will have several other team members prepared and available to fill in whenever needed and will do the work to onboard and train different testers for you. 

Of course, we speak from experience when it comes to outsourcing testing, as it’s what our team of over 100 testers are committed to delivering.

Recommended Reading: 6 Software Testing Outsourcing Myths Debunked

— 

These are just some of the biggest things that come to our mind regarding optimizing test spend. Before making any of these changes, make sure to communicate with your team so that everyone’s on board. 

If you had to think about how to be more impactful with your testing with fewer resources, how would you go about it? Let us know!

Contact us today for a free consultation to learn how you can reduce the cost of software testing.

---

Recommended for You

How to Choose a Software Testing Company
How to Quickly Set Up Test Automation in CI/CD