Getting started with test automation is no easy feat, but here are some tips to pull it off effectively

They say “automating chaos just gives you faster chaos,” not only faster, but (to paraphrase a song by Daft Punk) harder, faster, stronger… chaos. Automation can be an awesome productivity booster within testing teams and a quality enhancer for your systems when used correctly. The key is to make sure that it is used the right way, which is the hardest part when starting out. Here we will look at the most common test automation challenges and how to overcome them.

Challenge 1: Receiving the Green Light from Management

As with any company department, employees are always asking for things that may or may not be allowed for in the budget. Testers may already know that automation offers both business and technical benefits (decrease time to market, increase test coverage and accuracy, decrease cost of testing per hour, find bugs earlier, etc), but how can testers convince the finance department and the Director of Software Development to allocate the necessary time and funds for implementing test automation?

“Test automation promises increased productivity and accuracy, which is where the business case must be made. The cost of a single defect … can offset the price of one or more tool licenses.” — Randall Rice

We agree with Randall Rice here that automation can pay for itself. To prove to management that the financial benefits are substantial, you can show them this simple breakdown we did of the ROI of test automation. In it we demonstrated how a team of 5 senior and 5 entry level testers could hypothetically reduce the cost of testing from $78/hour to just $17.58/hour and increase testing from 1,350 hours per month to 5,985 equivalent hours, gaining $315,000 worth of testing via automation. Not to mention all of the qualitative benefits of automation.

It is important to also be very transparent with any and all stakeholders. Don’t lie to them and say that automation doesn’t require much upfront effort and resources. 

Challenge 2: Selecting and Using the Appropriate Tools

Many teams do not get past this phase due to several reasons. They may lack the expertise to use a certain tool, the tool they want doesn’t exist, their set of tools doesn’t offer complete test coverage, etc.

If you don’t have a sufficient base knowledge for how to use a tool, you have a few options:

• Take an online course
• Have someone that helped create the tool come and give training sessions
• Hire a consultant to help you master it
• When all else fails, outsourcing can be a practical approach 

If you think a tool doesn’t exist, it might be good to confirm it with the testing community. Go on to forums like Stack Exchange where fellow testers love to discuss developments in testing. If you can’t find it, you might want to see if it’s feasible or worthwhile to create it yourself. In our case, we have developed several custom testing tools for clients with specific needs. 

If the tool you have doesn’t do everything you need, consider finding a multi-tool solution. Remember, it’s impossible to test absolutely everything, but you can use the tools that test the most important things.

Lastly, if a tool is out of your budget, do a quick cost vs. benefit analysis and present your case. You can measure the damage done by a previous bug you have encountered and show how much time and money you could have saved if you had the tool in place. Again, you can also present them the information in our post, the ROI of Test Automation.

Challenge 3: Identifying a Starting Strategy

Ok, so you might have all the tools and the support to begin automating, but what do you actually automate and how? Unfortunately, the tools themselves do not tell you what to automate, just as new parents find to their dismay that children do not come with a handbook. Will you raise a generation of outstanding automated tests or will they turn out to be spoiled and unmanageable? Of course, you’d hope for the former! In reality, you can’t automate everything so you have to be strategic. You can use two approaches to help with this: risk-based testing and the automation pyramid.

Risk-Based Testing

Risk-based testing gives higher priority to testing the elements that are most at risk of failing which also carry the greatest negative consequences if said failure occurs. Here you should take into consideration:

• The financial impact of potential errors
• The probability of failure (here it is a good idea to ask the developers what they think)
• Service level agreements (SLA)
• Is there money or are there lives at stake? (Yes, this may seem dramatic, but there are several systems that deal with such important matters.)

This should give you a good method for prioritizing which test cases to automate.

The Test Automation Pyramid

Another approach that we highly recommend is to follow the automation pyramid. We wrote much more extensively about this topic in a recent post, but here is a quick overview.

The ice cream cone is sweet and tempting, but it could spoil your appetite for automation! Following the ice cream cone approach will result in high levels of frustration because it emphasizes automation on the UI level, which employs more brittle tests that break easily. Whereas, if you focus on automated unit tests, you are helping to prevent bugs or eliminate them almost immediately as you go through the software development lifecycle.

Challenge 4: Setting Realistic Expectations of Automation

No matter how great your tools and processes are, it’s important to remember that testing is never complete. Test automation is not a panacea for bug laden systems and shouldn’t be used in place of, but in conjunction with non-automated tests. There are some tests that simply can’t be automated, but there are some automated tests that uncover bugs which couldn’t be found otherwise.

We agree with the test automation philosophy of James Bach and Michael Bolton. Test automation is really just automatically checking the system, while humans are still needed to do the non-automated testing. Also, remember that the value of a test comes from the information that it provides, not the number of tests executed, nor the frequency. What we care most about is if we are getting the right information so that we can make the best possible decisions when improving the quality of our systems.

Make sure your team and management agree on and understand the desired outcome(s) from your automation plan so that everyone is on the same page! 

Thanks for reading! I hope this helps you automate tests more effectively.

Recommended for You

They Say Automation Increases Test Coverage, But What is Test Coverage Exactly?
How to Optimize Test Coverage in the Long-term