Blog

Webinar Recap: The State of Test Automation 2020 – Testim Survey Results

by October 29, 2020
Software TestingTest Automation
0

COVID-19’s impact on test automation, today’s biggest hurdles, and looking ahead

state of test automation webinar screenshot

In June, our friends at Testim conducted a survey on end-to-end test automation. So, what were the results?

Watch this webinar recording where our COO, Federico Toledo, joined Testim CEO Oren Rubin to uncover what developers, QA, and managers are saying about test automation.

They discussed the results of the survey and compared it to other data points gathered from related industry sources and their own experiences.

Webinar Highlights:

  • How has COVID-19 affected test automation?
  • What are the top challenges and priorities for automation
  • What do we expect in the future?

Watch the Webinar

TL;DW (Too Long, Didn’t Watch)?

Who Took the Survey?

To start off, here’s a breakdown of the survey respondents by role:

  • 46% – Automation Engineers
  • 18% – Developers / Dev managers
  • 16% – QA Testers
  • 13% – QA Managers
  • 7% – Other

And by team size:

  • 29% – 100 or more
  • 78.1% – Less than 100

What’s maybe even more interesting, is how the respondents answered they organized their teams:

From our experience, we were not too surprised to see these results, since in most of our own projects, we integrate our testers with our client’s development team, whether they’re using Scrum, or kanban, etc. We see it’s highly valuable to have the testers participate in the team’s daily calls, retrospectives, plannings, and even in product demos for the customer. This helps everyone in development to remain aligned and work towards a common goal.

What Skills Are Most Critical for Testers?

Coming in at number one for most important skills was test design and architecture, which makes it seem that the respondents emphasized the need for coding skills. Oren Rubin agreed that this was very important: being able to design and implement tests is critical.

Federico also wanted to place emphasis on the need for the human side of testing, looking beyond the technical answer choices that were provided:

“Critical thinking, communication, empathy are needed [in testing]. I wouldn’t trust the quality of product to someone who behaves like a scripting machine. I need someone engaged with the business, the organization…”

FEDERICO TOLEDO

What Are Your Biggest Challenges in Test Automation?

The fact that most people complained that test maintenance is one of the greatest challenges they face when it comes to test automation didn’t surprise the Testim team (given that they built a tool to help solve for that!). The next biggest pain that the survey indicated was the long time it takes to write UI functional tests.

Trends Impacting Test Automation in 2020

The top trends that respsondents indicated around test automation today were:

  • CI/CD – 55.6%
  • AI-assisted Testing – 46%
  • Continuous Testing – 38.1%
  • Shifting Left – 33.3%
  • Agile Testing – 55.4%
  • Testing in production – 25.4%

As Federico commented, these answers are consistent with our own experiences, as the shift-left testing approach becomes more popular, we see more engagement from different team members, taking part in automation, whether its unit testing or API testing. So these methodologies that are gaining more traction impact the way we automate and of course, this relates to CI/CD because we’re able to benefit more from automation since we’re using it in different stages before each release. If we’re continuously testing, then we should be shifting left and right at the same time, so these trends are fairly related to one another.

Automation Goals for the Future

Most respondents were looking to increase how many cross browser tests they run while many were satisfied with their current suite.

Oren commented that some of those who do not aim to grow their cross browser testing may be shooting for increasing their test coverage in a specific browser first. It may also be due to the high level of maintenance involved.

For teams that haven’t managed to already, it is clear that common goals include more automating of tests, integrating test automation with CI/CD, and increasing test coverage.

Federico commented that instead of making more tests, maybe it’s smarter to focus on making current tests more stable and more reliable.

What’s probably more interesting is where testers think the focus will shift in the next three years? From the results, it would seem that most expect to have already implemented CI/CD, with shifting left also losing importance as a priority. However, almost 80% answered AI-assisted testing will be a major trend looking ahead.

COVID-19’s Impact on Test Automation

And last but not least, we couldn’t fully understand today’s state of test automation without mentioning COVID-19. The survey touched on how test automation has been impacted by the pandemic. According to the results, two-thirds (67%), responded that it did not impact their team.

Conversely, as for their budget, over three-fourths responded that it did cause their budgets to be reduced.

Oren speculated that the respondents must have had two different budgets in mind while going through the questions: one for people and the other for tooling. Teams may have invested in more tooling to compensate for not being able to hire more development talent during COVID-19 as their goals most likely still remained the same.

For us at Abstracta, one of the main impacts of COVID-19 was an acceleration in cultural shift. Nowadays, we see that our clients and other companies are more ready to accept the possibility that teams can work seamlessly in a distributed manner. As a benefit of this, companies can source the brightest talent, wherever it may be. We have been forced to learn more communication tools during the pandemic, which can help us to work in a more fluid, and even, asynchronous way.

As for COVID-19’s impact on team’s test automation initiatives, 45% answered that they had been slowed down, 40%, accelerated, and 15% reported no impact.

What do you think of these survey results on the current state of test automation? Anything surprise you? Let us know in the comments!

P.S., Major kudos to Tristan, Testim’s Community Manager for live tweeting the webinar!

If your team needs some extra assistance this year to successfully automate tests, our remote automation engineers can help!

Recommended for You

16 Software Testing Podcasts to Listen to in 2020
Improve Your Software Testing Strategy: A Maturity Model

Tags In
COVID-19 Crossbrowser Testing Testim Webinar
203 / 437

Related Posts