{"id":3222,"date":"2015-08-31T02:52:51","date_gmt":"2015-08-31T02:52:51","guid":{"rendered":"http:\/\/www.abstracta.us\/?p=3222"},"modified":"2025-05-05T21:21:36","modified_gmt":"2025-05-05T21:21:36","slug":"true-roi-test-automation","status":"publish","type":"post","link":"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/","title":{"rendered":"The True ROI of Test Automation"},"content":{"rendered":"<p><!-- Go to www.addthis.com\/dashboard to customize your tools --><script src=\"\/\/s7.addthis.com\/js\/300\/addthis_widget.js#pubid=ra-58d80a50fc4f926d\" type=\"text\/javascript\"><\/script><\/p>\n<h1><span style=\"font-weight: 400; color: #333333;\"> Is test automation just a huge\u00a0expense or an investment? <\/span><\/h1>\n<p><span style=\"font-weight: 400; color: #333333;\">Is there a positive<strong> ROI of test automation?<\/strong>\u00a0Generally, it\u2019s said that automated testing can extend the coverage and scope of testing, reduce costs, mitigate risk, put the focus on manual testing where it\u2019s really necessary, find defects earlier, etc. etc. This seems to always be the same spiel that tool vendors pose without much proof.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">While many testers themselves know that there\u2019s a multitude of benefits associated with <a href=\"https:\/\/abstracta.us\/solutions\/test-automation\">test automation<\/a>, one of the number one challenges of getting started is obtaining management buy-in and commitment. Luckily, there is a way to convince managers that there\u2019s something in it for them as well. Today we\u2019ll discuss not only the ways in which test automation is valuable for software development, but also the ways in which it provides managers and financial executives real business value.<\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"When_Test_Automation_is_Useful\"><\/span><strong><span style=\"color: #00b674;\">When Test Automation is Useful<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\">The best\u00a0uses for test automation involve tests that repeatable and are necessary to be run many times (either because the app will have many versions, for maintenance reasons, or because it must be tested on different platforms). One such example of this type of test is regression tests. Other tests that should be considered for automation include those that cover high-risk functionalities that, if they were to fail, would cause a great negative impact to the business.\u00a0<\/span><\/p>\n<p>For more on which tests to automate, <a href=\"https:\/\/abstracta.us\/blog\/test-automation\/when-to-automate-a-test\/\" target=\"_blank\" rel=\"noopener noreferrer\">read this post<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Manual_vs_Automated_Testing\"><\/span><strong><span style=\"color: #00b674;\">Manual vs Automated Testing<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\">Not everything can be automated, and there\u2019s no need to try to replace all manual testing because there are things that just can\u2019t be automated and are not worth automating. Testing a new functionality manually allows you to quickly know more about the application at a low cost. As knowledge accumulates, the inventory of tests increases, and consequently, the cost also increases\u00a0of manual testing. On the other hand, automation has a higher initial cost which decreases as it progresses. This behavior can be seen in the following graph (<a href=\"http:\/\/blog.qatestlab.com\/2018\/06\/12\/when-automate-testing\/\" target=\"_blank\" rel=\"noopener noreferrer\">taken from here<\/a>):<\/span><\/p>\n<p><img decoding=\"async\" class=\"aligncenter wp-image-11158\" src=\"http:\/\/abstracta.us\/wp-content\/uploads\/2019\/04\/manual-automation-ratio-min-300x184.jpg\" alt=\"automated vs manual testing costs\" width=\"500\" height=\"307\" \/><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Benefits\"><\/span><strong><span style=\"color: #00b674;\">Benefits<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\">Automated tests run on average 5 times faster (probably many more times than that, but we\u2019ll be conservative here) than manual tests. If we automate the tests of a development cycle, we\u2019ll be able to re-test with minimal effort during the next cycle, allowing us to increase the size of the test suite, thus expanding coverage. Manual testing lets you find the few critical defects that are easy to locate while automated testing lets you go beyond that to find subtle defects that you are unlikely to find. Lastly, If we don\u2019t automate, we may end up with test cycles that are longer than development cycles, or decide not to test certain things at all, subjecting us to risk.<\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Whats_the_ROI_of_Test_Automation\"><\/span><strong><span style=\"color: #00b674;\">What&#8217;s the ROI of Test Automation?\u00a0<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><a href=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/08\/show-me-the-ROI-resized-600.jpeg-compressor-1.png\" rel=\"attachment wp-att-4445\"><img decoding=\"async\" class=\"wp-image-4447 aligncenter\" src=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/08\/show-me-the-ROI-resized-600.jpeg-compressor-1-300x162.png\" alt=\"Show me the ROI!\" width=\"563\" height=\"304\" \/><\/a><\/p>\n<p>A manual tester runs tests 8 hours per day and goes home. At that time, testing stops. With test automation, we can run tests 16 more hours a day (in the best case scenario&#8230; of course) for the same cost, reducing the average cost of testing hours.<\/p>\n<p><span style=\"font-weight: 400; color: #000000;\"><span style=\"color: #333333;\">Here, we will look at the case example from Paul Grossman\u2019s<\/span><span style=\"color: #333333;\"><span style=\"color: #000000;\">\u00a0<a href=\"https:\/\/docplayer.net\/20448512-Table-of-contents-automated-testing-roi-fact-or-fiction-a-customer-s-perspective-what-real-qa-organizations-have-found.html\" target=\"_blank\" rel=\"noopener noreferrer\">white paper<\/a>,\u00a0<\/span><\/span><\/span><span style=\"font-weight: 400; color: #000000;\"><span style=\"color: #333333;\">\u201cAutomated Testing ROI: Fact or Fiction?\u201d<\/span><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Consider the case of practicing manual testing only. If a tester on average costs $50 an hour and if a senior tester who creates automated tests costs $75 an hour, that would cost about $400 and $600 respectively per day per tester.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Now, consider a team of 10 testers, five senior-level and five entry-level, with a monthly loaded cost of $105,000 (for 168 hours per month). We\u2019d get a total of 1,350 hours costing $78.00\/ hour (this is assuming each tester realistically works 135 hours per month due to breaks, training days, vacations, etc.). If we automate testing, the cost of labor would remain the same, but for the effort of 3 test automation engineers, we\u2019d achieve \u00a016 hours a day of testing and will run 5x more tests per hour. This results in the equivalent of 5,040 hours per month of manual testing created by the three test automation engineers. Then, consider the rest of the team doing manual testing (7 people x 135 hours\/month). That amounts to 945 hours more, ending with a combined total of 5,985 hours of testing at $17,54\/hour ($105,000 divided by 5,985 hours).<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\"><a style=\"color: #333333;\" href=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/09\/Screen-Shot-2015-08-28-at-12.01.24-PM.png\"><img decoding=\"async\" class=\"wp-image-3225 aligncenter\" src=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/09\/Screen-Shot-2015-08-28-at-12.01.24-PM.png\" alt=\"Screen Shot 2015-08-28 at 12.01.24 PM\" width=\"416\" height=\"214\" \/><\/a><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">In this scenario, we\u2019ve\u00a0<strong>dramatically reduced the cost of each test hour from $78 to $17.54,<\/strong>\u00a0which is a benefit that the CFO will clearly understand. Or you could look at it this way; we have\u00a0<strong>increased testing from 1,350 hours to 5,985 equivalent hours and gained $315,000 worth of testing per month<\/strong>\u00a0for the same cost (5,040 times the average hourly cost of a tester).<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Not only do we test quicker, but the test coverage is larger, which means we can find more bugs! But, finding bugs certainly means we will have more work to do and boatloads more money needed to fix them right? Not necessarily.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">It costs much less to fix bugs that are detected earlier in the development cycle. In the chart below, you can see the cost of correcting a defect detected by the stage in which it has been found (development, integration, beta testing, or production). We will assume that it costs $75\/hour to fix bugs. These bug costs don\u2019t include hidden ones as well such as loss of reputation, trust, and even equipment wear.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\"><a style=\"color: #333333;\" href=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/09\/Screen-Shot-2015-09-20-at-7.43.38-PM.png\"><img decoding=\"async\" class=\"aligncenter wp-image-3228 \" src=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/09\/Screen-Shot-2015-09-20-at-7.43.38-PM.png\" alt=\"Screen Shot 2015-09-20 at 7.43.38 PM\" width=\"482\" height=\"221\" \/><\/a><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Data Source: <em>(Planning Report 02-3, \u201cThe Economic Impacts of Inadequate Infrastructure for Software Testing,\u201d Prepared by RTI for National Institute of Standards and Technology, May 2002, p 7-12.).<\/em><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">As you can see, the sooner we find bugs, the cheaper and easier it is to fix them. If we practice test automation, it&#8217;s more probable that we will find more bugs before the beta testing and production phases. It\u2019s difficult to estimate how much, but in general for every bug that we find in the early stages, we will save $200 (not bad)!\u00a0<strong>Coding defects found post-release cost 5x more to fix than those found during unit testing.<\/strong><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">It\u2019s safe to say that there is a high <strong>ROI of test automation<\/strong> and that it is a GOOD investment because it provides value in two ways:<\/span><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Business_Value\"><\/span><strong><span style=\"color: #333399;\">Business Value<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul>\n<li><span style=\"font-weight: 400; color: #333333;\">Improve software quality<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Avoid operational problems<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Maintain a good customer image<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Avoid legal problems<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Decrease cost of fixing bugs by 5x<\/span><\/li>\n<\/ul>\n<h3><span class=\"ez-toc-section\" id=\"IT_Value\"><\/span><strong><span style=\"color: #333399;\">IT\u00a0Value<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul>\n<li><span style=\"font-weight: 400; color: #333333;\">Simplifies routine tasks<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Run more tests without increasing costs in the same amount of time<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Increase the scope of coverage<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Find the hard-to-detect defects earlier, when they are easier to fix<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Improve Software Quality<\/span><\/li>\n<\/ul>\n<p>Do you agree that the ROI of test automation is significant? Comment below!<\/p>\n<p><strong>Download our ebook, <a href=\"https:\/\/abstracta.us\/functional-test-automation-ebook\">A Complete Introduction to Functional Test Automation<\/a>:<\/strong><\/p>\n<p><img decoding=\"async\" class=\"aligncenter wp-image-10934 size-full\" src=\"http:\/\/abstracta.us\/wp-content\/uploads\/2015\/11\/ebook-blog-abstracta-1-min.png\" alt=\"read our functional test automation ebook\" width=\"500\" height=\"320\" \/><\/p>\n<hr \/>\n<h2><span class=\"ez-toc-section\" id=\"Recommended_for_You\"><\/span><span style=\"font-weight: 400;\">Recommended for You<\/span><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/beginners-guide-functional-test-automation\/\">The Complete Beginner&#8217;s Guide to Functional Test Automation<\/a><\/span><br \/>\n<span style=\"color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/4-common-test-automation-challenges-overcome\/\">The 4 Most Common Test\u00a0Automation Challenges (And How to Overcome Them)<\/a>\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Is test automation just a huge\u00a0expense or an investment? Is there a positive ROI of test automation?\u00a0Generally, it\u2019s said that automated testing can extend the coverage and scope of testing, reduce costs, mitigate risk, put the focus on manual testing where it\u2019s really necessary, find&#8230;<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[60],"tags":[68],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v14.0.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>What&#039;s the True ROI of Test Automation? | Abstracta<\/title>\n<meta name=\"description\" content=\"Here, we will look at the case example from Paul Grossman\u2019s white paper, \u201cAutomated Testing ROI: Fact or Fiction?\u201d to assess the ROI of test automation.\" \/>\n<meta name=\"robots\" content=\"index, follow\" \/>\n<meta name=\"googlebot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta name=\"bingbot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What&#039;s the True ROI of Test Automation? | Abstracta\" \/>\n<meta property=\"og:description\" content=\"Here, we will look at the case example from Paul Grossman\u2019s white paper, \u201cAutomated Testing ROI: Fact or Fiction?\u201d to assess the ROI of test automation.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/AbstractaQA\/\" \/>\n<meta property=\"article:published_time\" content=\"2015-08-31T02:52:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-05T21:21:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/abstracta.us\/wp-content\/uploads\/2015\/08\/True_ROI-min.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"420\" \/>\n\t<meta property=\"og:image:height\" content=\"236\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@AbstractaUS\" \/>\n<meta name=\"twitter:site\" content=\"@AbstractaUS\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/abstracta.us\/blog\/#website\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\",\"description\":\"AI-powered quality engineering\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/abstracta.us\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"http:\/\/abstracta.us\/wp-content\/uploads\/2019\/04\/manual-automation-ratio-min-300x184.jpg\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/#webpage\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\",\"name\":\"What's the True ROI of Test Automation? | Abstracta\",\"isPartOf\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/#primaryimage\"},\"datePublished\":\"2015-08-31T02:52:51+00:00\",\"dateModified\":\"2025-05-05T21:21:36+00:00\",\"author\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\"},\"description\":\"Here, we will look at the case example from Paul Grossman\\u2019s white paper, \\u201cAutomated Testing ROI: Fact or Fiction?\\u201d to assess the ROI of test automation.\",\"breadcrumb\":{\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/\",\"name\":\"Test Automation\"}},{\"@type\":\"ListItem\",\"position\":3,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\",\"name\":\"The True ROI of Test Automation\"}}]},{\"@type\":[\"Person\"],\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\",\"name\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/abstracta.us\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/222e8b1136482564fe25acc4de2b9b7a?s=96&d=blank&r=g\",\"caption\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\"},\"description\":\"Co-Chief Executive Officer at Abstracta\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3222"}],"collection":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/comments?post=3222"}],"version-history":[{"count":24,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3222\/revisions"}],"predecessor-version":[{"id":14268,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3222\/revisions\/14268"}],"wp:attachment":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/media?parent=3222"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/categories?post=3222"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/tags?post=3222"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}