{"id":3914,"date":"2015-11-16T08:24:59","date_gmt":"2015-11-16T08:24:59","guid":{"rendered":"http:\/\/www.abstracta.us\/?p=3914"},"modified":"2025-05-05T21:21:28","modified_gmt":"2025-05-05T21:21:28","slug":"4-common-test-automation-challenges-overcome","status":"publish","type":"post","link":"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/","title":{"rendered":"The 4 Most Common Test Automation Challenges (and How to Overcome Them)"},"content":{"rendered":"<p><!-- Go to www.addthis.com\/dashboard to customize your tools --><script src=\"\/\/s7.addthis.com\/js\/300\/addthis_widget.js#pubid=ra-58d80a50fc4f926d\" type=\"text\/javascript\"><\/script><\/p>\n<h1><span style=\"font-weight: 400; color: #333333;\">Getting started with test automation\u00a0is no easy feat, but here are some tips to pull it off effectively<\/span><\/h1>\n<p><span style=\"color: #333333;\"><span style=\"font-weight: 400;\">They say &#8220;automating chaos just gives you faster chaos,&#8221; not only faster, but (to paraphrase a song by Daft Punk) harder, faster, stronger&#8230; chaos. Automation can be an awesome productivity booster within testing teams and a quality enhancer for your systems when used <em>correctly<\/em>. The key is to make sure that it is used the right way, which is the hardest part when starting out. <\/span><span style=\"color: #333333;\"><span style=\"font-weight: 400;\">Here we will look at the most common<b> test automation challenges<\/b> and how to overcome them. <\/span><\/span><\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Challenge_1_Receiving_the_Green_Light_from_Management\"><\/span>Challenge 1: Receiving the Green Light from Management<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\">As with any company department, employees are always asking for things that may or may not be allowed for in the budget. Testers may already know that automation offers both business and\u00a0technical benefits (decrease time to market, increase test coverage and accuracy, decrease cost of testing per hour, find bugs earlier, etc), but how can testers convince the finance department and the Director of Software Development to allocate the necessary time and funds for implementing test automation? <\/span><\/p>\n<p><span style=\"color: #333333;\"><i><span style=\"font-weight: 400;\">\u201cTest automation promises increased productivity and accuracy, which is where the business case must be made. The cost of a single defect &#8230; can offset the price of one or more tool licenses.\u201d &#8212; Randall Rice<\/span><\/i><\/span><\/p>\n<p><span style=\"color: #000000;\"><span style=\"font-weight: 400; color: #333333;\">We agree with Randall Rice here that automation can pay for itself. To prove to management that the financial benefits are substantial, you can show them this simple breakdown we did of the <\/span><span style=\"color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">ROI of test automation<\/span><\/a><\/span><span style=\"font-weight: 400; color: #333333;\">. In it we demonstrated how a team of 5 senior and 5 entry level testers could hypothetically reduce the cost of testing from $78\/hour to just $17.58\/hour and increase testing from 1,350 hours per month to 5,985 equivalent hours, gaining $315,000 worth of testing via automation. Not to mention all of the qualitative benefits of automation. <\/span><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">It is important to also be very transparent with any and all stakeholders. Don\u2019t lie to them and say that automation doesn\u2019t require much\u00a0upfront effort and resources.\u00a0<\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Challenge_2_Selecting_and_Using_the_Appropriate_Tools\"><\/span>Challenge 2: Selecting and Using the Appropriate Tools<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\">Many teams do not get past this phase due to several reasons. They may lack the expertise to use a certain tool, the tool they want doesn\u2019t exist, their set of tools doesn&#8217;t offer\u00a0complete test coverage,\u00a0etc.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">If you don\u2019t have a sufficient base knowledge for how to use a tool, you have a few options:<\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400; color: #333333;\">Take an online course<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Have someone that helped create the tool come and give training sessions<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Hire a consultant to help you master it <\/span><\/li>\n<li><span style=\"font-weight: 400; color: #000000;\"><span style=\"color: #333333;\">When all else fails,<\/span> <span style=\"color: #000080;\"><span style=\"color: #333333;\"><a href=\"http:\/\/abstracta.us\/blog\/software-testing\/getting-started-outsourced-software-testing\/\">outsourcing<\/a> can be a practical approach\u00a0<\/span><\/span><\/span><\/li>\n<\/ul>\n<p><span style=\"color: #000000;\"><span style=\"font-weight: 400; color: #333333;\">If you think a tool doesn\u2019t exist, it might be good to confirm it with the testing community. Go on to forums like <\/span><span style=\"color: #00b674;\"><a href=\"http:\/\/sqa.stackexchange.com\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Stack Exchang<\/span><span style=\"font-weight: 400;\">e<\/span><\/a><\/span><span style=\"font-weight: 400;\"><span style=\"color: #333333;\"> where fellow testers love to discuss developments in testing. If you can\u2019t find it, you might want to see if it\u2019s feasible or worthwhile to create it yourself. In our case, we have\u00a0developed several\u00a0<a href=\"https:\/\/abstracta.us\/solutions\/custom-software-testing-tool-development\" target=\"_blank\" rel=\"noopener\">custom testing tools<\/a>\u00a0for clients with specific needs.\u00a0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-weight: 400;\">If the tool you have doesn\u2019t do everything you need, consider finding a multi-tool solution.<\/span><span style=\"font-weight: 400;\">\u00a0Remember, it\u2019s impossible to test absolutely everything, but you can use the tools that test the most important things. <\/span><\/span><\/p>\n<p><span style=\"color: #000000;\"><span style=\"font-weight: 400; color: #333333;\">Lastly, if a tool is out of\u00a0your budget, do a quick cost vs. benefit analysis and present your case. You can measure the damage done by a previous bug you have encountered and show how much time and money you could have saved if you had the tool in place. Again, you can also present them the information in our post, the <\/span><span style=\"color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">ROI of Test Automation<\/span><\/a><\/span><span style=\"font-weight: 400; color: #333333;\">.<\/span><\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Challenge_3_Identifying_a_Starting_Strategy\"><\/span>Challenge 3: Identifying a Starting Strategy<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"color: #333333;\"><span style=\"font-weight: 400;\">Ok, so you might have all the tools and the support to begin automating, but what do you actually automate and how? Unfortunately, the tools themselves do not tell you what to automate, just as new parents find to their dismay that children do not come with a handbook. Will you raise a generation of outstanding automated tests or will they turn out to be spoiled and unmanageable? Of course, you\u2019d hope for the former! In reality, you can\u2019t automate everything so you have to be strategic. You can use two approaches to help with this: <\/span><b>risk-based testing and the automation pyramid<\/b><span style=\"font-weight: 400;\">.<\/span><\/span><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Risk-Based_Testing\"><\/span><strong>Risk-Based Testing<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"font-weight: 400; color: #333333;\">Risk-based testing gives higher priority to testing the elements that are most at risk of failing which also carry the greatest negative consequences if said failure occurs. Here you should take into consideration: <\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400; color: #333333;\">The financial impact of potential errors<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">The probability of failure (here it is a good idea to ask the developers what they think)<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Service level agreements (SLA)<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Is there money or are there lives at stake? (Yes, this may seem dramatic, but there are several systems that deal with such important matters.)<\/span><\/li>\n<\/ul>\n<p><span style=\"color: #000000;\"><span style=\"font-weight: 400; color: #333333;\">This should give you a good method for prioritizing which test cases to automate. <\/span><\/span><\/p>\n<h3><span class=\"ez-toc-section\" id=\"The_Test_Automation_Pyramid\"><\/span><strong>The Test Automation Pyramid<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"color: #000000;\"><span style=\"font-weight: 400; color: #333333;\">Another approach that we highly recommend is to follow the automation pyramid. We wrote much more extensively about this topic in a <\/span><span style=\"color: #00b674;\"><a href=\"https:\/\/abstracta.us\/blog\/test-automation\/best-testing-practices-for-agile-teams-the-automation-pyramid\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">recent post<\/span><\/a><\/span><span style=\"font-weight: 400;\">, <span style=\"color: #333333;\">but here<\/span> is a quick overview.<\/span><\/span><\/p>\n<p><a href=\"http:\/\/abstracta.us\/wp-content\/uploads\/2015\/10\/Screen-Shot-2017-03-27-at-6.21.09-PM-min-1.png\"><img decoding=\"async\" class=\"aligncenter wp-image-8840\" src=\"http:\/\/abstracta.us\/wp-content\/uploads\/2015\/10\/Screen-Shot-2017-03-27-at-6.21.09-PM-min-1-300x156.png\" alt=\"test automation pyramid\" width=\"650\" height=\"338\" \/><\/a><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">The ice cream cone is sweet and tempting, but it could spoil your appetite for automation! Following the ice cream cone approach will result in high levels of frustration because it emphasizes automation on the UI level, which employs more brittle tests that break easily. Whereas, if you focus on automated unit tests, you are helping to prevent bugs or eliminate them almost immediately as you go through the software development lifecycle. <\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Challenge_4_Setting_Realistic_Expectations_of_Automation\"><\/span>Challenge 4: Setting Realistic Expectations of Automation<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #000000;\"><span style=\"color: #333333;\">No matter how great your tools and processes are, it\u2019s important to remember that testing is never complete. Test automation is not a panacea for bug laden systems and shouldn\u2019t be used in place of, but in conjunction with non-automated tests. There are some tests that simply can\u2019t be automated, but there are some automated tests that uncover bugs which couldn\u2019t be found otherwise.<\/span> <\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">We agree with the <\/span><a href=\"http:\/\/www.satisfice.com\/blog\/archives\/856\" target=\"_blank\" rel=\"noopener\">test automation philosophy<\/a><span style=\"font-weight: 400; color: #333333;\"> of James Bach and Michael Bolton. Test automation is really just automatically checking the system, while humans are still needed to do the non-automated testing. <\/span><span style=\"font-weight: 400; color: #333333;\">Also, remember that the value of a test comes from the information that it provides, not the number of tests executed, nor the frequency. What we care most about is if we are getting the right information so that we can make the best possible decisions when improving the quality of our systems.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Make sure your team and management agree on and understand the desired outcome(s) from your automation plan so that everyone is on the same page!\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Thanks for reading! I hope this helps you automate tests more effectively.<\/span><\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h2><span class=\"ez-toc-section\" id=\"Recommended_for_You\"><\/span><strong>Recommended for You<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/\" target=\"_blank\" rel=\"noopener\">They Say Automation Increases Test Coverage, But What is Test Coverage Exactly?<\/a><\/span><br \/>\n<span style=\"font-weight: 400; color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/optimize-test-coverage-long-term\/\" target=\"_blank\" rel=\"noopener\">How to Optimize Test Coverage in the Long-term<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Getting started with test automation\u00a0is no easy feat, but here are some tips to pull it off effectively They say &#8220;automating chaos just gives you faster chaos,&#8221; not only faster, but (to paraphrase a song by Daft Punk) harder, faster, stronger&#8230; chaos. Automation can be&#8230;<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[60],"tags":[68],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v14.0.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The 4 Most Common Test Automation Challenges (and How to Overcome Them) | Abstracta<\/title>\n<meta name=\"description\" content=\"They say &quot;automating chaos just gives you faster chaos.&quot; Here we will look at the most common test automation challenges and how to overcome them.\" \/>\n<meta name=\"robots\" content=\"index, follow\" \/>\n<meta name=\"googlebot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta name=\"bingbot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The 4 Most Common Test Automation Challenges (and How to Overcome Them) | Abstracta\" \/>\n<meta property=\"og:description\" content=\"They say &quot;automating chaos just gives you faster chaos.&quot; Here we will look at the most common test automation challenges and how to overcome them.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/AbstractaQA\/\" \/>\n<meta property=\"article:published_time\" content=\"2015-11-16T08:24:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-05T21:21:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/abstracta.us\/wp-content\/uploads\/2015\/11\/Test-Automation-Challenge-min.png\" \/>\n\t<meta property=\"og:image:width\" content=\"560\" \/>\n\t<meta property=\"og:image:height\" content=\"315\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@AbstractaUS\" \/>\n<meta name=\"twitter:site\" content=\"@AbstractaUS\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/abstracta.us\/blog\/#website\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\",\"description\":\"AI-powered quality engineering\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/abstracta.us\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"http:\/\/abstracta.us\/wp-content\/uploads\/2015\/10\/Screen-Shot-2017-03-27-at-6.21.09-PM-min-1-300x156.png\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/#webpage\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/\",\"name\":\"The 4 Most Common Test Automation Challenges (and How to Overcome Them) | Abstracta\",\"isPartOf\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/#primaryimage\"},\"datePublished\":\"2015-11-16T08:24:59+00:00\",\"dateModified\":\"2025-05-05T21:21:28+00:00\",\"author\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\"},\"description\":\"They say \\\"automating chaos just gives you faster chaos.\\\" Here we will look at the most common test automation challenges and how to overcome them.\",\"breadcrumb\":{\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/\",\"name\":\"Test Automation\"}},{\"@type\":\"ListItem\",\"position\":3,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/\",\"name\":\"The 4 Most Common Test Automation Challenges (and How to Overcome Them)\"}}]},{\"@type\":[\"Person\"],\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\",\"name\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/abstracta.us\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/222e8b1136482564fe25acc4de2b9b7a?s=96&d=blank&r=g\",\"caption\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\"},\"description\":\"Co-Chief Executive Officer at Abstracta\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3914"}],"collection":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/comments?post=3914"}],"version-history":[{"count":28,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3914\/revisions"}],"predecessor-version":[{"id":17530,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3914\/revisions\/17530"}],"wp:attachment":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/media?parent=3914"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/categories?post=3914"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/tags?post=3914"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}