How to apply test automation efficiently and effectively with the automation pyramid

Test automation and agile software development go hand in hand, but automating is often easier said than done. Most developers recognize the benefits of test automation: it speeds up testing, lowers costs, increases coverage, etcetera, but, many never get past the initial investment required to get it started. Like the cavemen in this cartoon, a lot of teams get stuck using less efficient testing methods because they think they don’t have the time to implement changes, when really, they are doing themselves a disservice. Don’t fall into this bad habit! 

Today we will share with you one of our best testing practices for agile teams.

How do we get started? How do we know on which areas to focus? Which test cases should be automated? In non-agile software development, many people end up inadvertently falling into the “ice cream cone anti-pattern” for testing by putting more emphasis on automating at the UI level. At Abstracta, we’re more fond of the approach that flips that ice cream cone upside down. We agree with the approach made popular by Mike Cohn, the agile test automation pyramid, that gives you the most bang for your automation buck, improving the ROI of automation and guaranteeing that you will receive the most benefits from automation.

When most of our efforts are focused on automation at the UI level, the focus is on finding bugs, whereas, with the agile pyramid, the idea is to prevent them.

In the figure below, you can see how the two approaches differ.

Base Layer: Unit Tests

Clearly in the pyramid, (as a part of the best testing practices for agile teams), most of the testing should take place in the development stage, running unit tests after every build. These tests are the easiest, cheapest, and fastest to complete and are an important aspect of test-driven development. Running more tests at a lower level allows us to “check our work” as we go, getting feedback immediately and allowing us to know exactly where the bugs are when it is much harder for them to hide. Here, the bugs will also have a shorter lifespan, having been born and removed in less than a minute, perhaps. During the UI tests, bugs will have lived for much longer and will put up a greater fight because they have lived there very comfortably for a longer period of time (perhaps even a couple of days).

Mid-layer: API/ Integration/ Component Tests

After we run all of the unit tests and they pass, we can move onto the API/ integration/ component testing phase. Integration tests are run to make sure that all the components work together properly. This is where we can test most of the logic and business processes without going through the UI. It is best to automate here as much as possible. If you have to decide whether to automate at this level or at the UI level, here you’ll have fewer problems, easier maintenance, faster test execution (meaning finding bugs sooner and decreasing their lifespans) and you get to test the logic of your system. These tests are slower and more complex than unit tests, but they are still faster and less brittle than UI tests.

Top Layer: UI Tests

Last and run least are UI tests. It’s best to run as few as possible as they are costly, more difficult to prepare and maintain, and take a long time. Here you just want to make sure that the user interface itself works properly, knowing that all the other aspects of the system should have already been tested. Automate only the most critical tests, end to end, with a flow starting from the user login and ending with their final action, like the transaction success message. It’s also helpful to focus on things related to the browsers or the UI. Be careful with these tests as they are more likely to provide false negatives and false positives. After running the UI tests, manual and exploratory testing can be conducted (as shown in the sphere shape above the pyramid). 

As you can see, the pyramid approach is a stronger, more beneficial and cost-effective way to implement test automation than putting the focus on automated GUI tests and inadvertently following the “ice cream cone anti-pattern.” The pyramid provides a strong base in the unit testing phase from which to build upon further testing in the integration and UI phases, whereas the ice cream cone approach is more “top heavy” and less stable.

To excel in the agile development world, it is imperative to follow the automation testing pyramid in order to produce the best quality software possible.

Don’t take just our word for it! Even Google’s testers say this is the best method.

Have you tried it before? Let us know in the comments!

Recommended for You

The 4 Most Common Challenges of Test Automation (And How to Overcome Them)
How to Avoid False Positives and False Negatives in Test Automation