{"id":3591,"date":"2015-10-19T22:58:42","date_gmt":"2015-10-19T22:58:42","guid":{"rendered":"http:\/\/www.abstracta.us\/?p=3591"},"modified":"2025-05-05T21:21:32","modified_gmt":"2025-05-05T21:21:32","slug":"say-automation-increases-test-coverage-test-coverage-exactly","status":"publish","type":"post","link":"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/","title":{"rendered":"They Say Automation Increases Test Coverage, but What Is Test Coverage Exactly?"},"content":{"rendered":"<p><!-- Go to www.addthis.com\/dashboard to customize your tools --><script src=\"\/\/s7.addthis.com\/js\/300\/addthis_widget.js#pubid=ra-58d80a50fc4f926d\" type=\"text\/javascript\"><\/script><\/p>\n<h1><span style=\"font-weight: 400; color: #333333;\">What you need to know about test coverage<\/span><\/h1>\n<p><span style=\"font-weight: 400; color: #333333;\">Recently, I posted a blog about the<\/span><span style=\"color: #00b674;\"> <a href=\"http:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\" target=\"_blank\" rel=\"noopener\">ROI of test automation<\/a> <\/span><span style=\"font-weight: 400; color: #333333;\">and its benefits. I mentioned that it increases test coverage, but do we really know <strong>what is test coverage<\/strong>? <\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">In this post, I\u2019ll go into more detail about it.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Basically, it\u2019s one of the measurements of test quality that tells us how much code has been tested. It defines certain entities of the system with the intent to cover them with tests. It\u2019s a way to indicate when we have tested sufficiently and gives us ideas of what else to test (thus<\/span><span style=\"font-weight: 400; color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/optimize-test-coverage-long-term\/\"> expanding coverage<\/a><\/span><span style=\"color: #333333;\">).<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Consider carrying out tests to be like sweeping the floors of your house. I always forget to sweep some part, like the upstairs bathroom, so my coverage does not include that room. Imagine if I only included sweeping bedrooms for my sweep coverage criteria.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\"><a href=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/10\/Test-Coverage-sweeper.jpg\" rel=\"attachment wp-att-4440\"><img decoding=\"async\" class=\" wp-image-4440 aligncenter\" src=\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/10\/Test-Coverage-sweeper-211x300.jpg\" alt=\"little sweeper\" width=\"343\" height=\"343\" \/><\/a><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\"><span style=\"font-weight: 400; color: #333333;\">With that criteria, if I sweep 100% of the bedrooms, does that mean that the whole house is clean?<\/span><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">No, because I completely missed the kitchen, dining room, bathrooms, etc! Therefore, be careful and measure test coverage with caution. To have a certain level of coverage is an indicator of the quality of the tests, but it is never an indicator of the quality of the system, neither does it guarantee that everything has been tested. Test coverage tells us what percentage of the code has been tested, but it doesn\u2019t mean that it has been tested in every situation either. <\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"What_is_Test_Coverage_Good_For\"><\/span>What is Test Coverage Good For?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\">In this case, test coverage is good for:<\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400; color: #333333;\">Measuring the quality of sweeping<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Indicating when to stop sweeping<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Alerting me of other places that need to be swept<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400; color: #333333;\">Certain criteria can be more powerful than other criteria. Knowing them can give you an indicator of how deep the tests are and when to apply one criterion or another. For example, it is said that a criteria A includes another criteria B if any set of test cases TS that covers criteria A also covers criteria B.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">For sweeping, we may want to follow the following criteria:<\/span><\/p>\n<ol>\n<li><span style=\"font-weight: 400; color: #333333;\">Sweep every bedroom<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Sweep every part of the house (bedroom, bathroom, kitchen, etc)<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Sweep every tiny spot, even the corners, because they are likely to accumulate dirt. <\/span><\/li>\n<\/ol>\n<p><span style=\"font-weight: 400; color: #333333;\">As you can see, Criterion 3 includes 2, which includes 1 (the relationship is transitive as number 3 includes number 1. If we design a test case for criterion number 3, it should also cover the first two criteria. For testing software, the criteria usually include the various paths, conditions, statements, functions, etc. within a program.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Another real example could be, for instance, equivalence class partitioning, where classes are defined and then one element from each class is selected, and in this way, we cover all the classes. If we think about white-box testing, we have sentence coverage, branch coverage, path coverage, etc., and in particular, for state machines, we have criteria that indicate to cover all nodes, all transitions, etc.<\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Where_Does_Test_Automation_Fit_Into_All_This\"><\/span>Where Does Test Automation Fit Into All This?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\"> Well, imagine you ditched the broom and replaced it with a super speedy robot. You would manage to clean the floors faster and miss fewer spots, also allowing you to focus on more important things while it does the work for you. <\/span><\/p>\n<p>[tweet_box design=&#8221;default&#8221; float=&#8221;none&#8221;]More test automation = More test coverage.[\/tweet_box]<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Do you measure test coverage? <\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">Can you imagine how test automation would help to improve this measurement?\u00a0<\/span><span style=\"color: #000000;\"><span style=\"font-weight: 400; color: #333333;\">Get started with\u00a0<a href=\"https:\/\/abstracta.us\/solutions\/test-automation\" target=\"_blank\" rel=\"nofollow noopener\">test automation<\/a><\/span> <span style=\"font-weight: 400; color: #333333;\">today!<\/span><\/span><\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h2><span class=\"ez-toc-section\" id=\"Recommended_for_You\"><\/span>Recommended for You<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/software-testing\/webinar-summary-use-automation-increase-test-coverage\/\">Webinar Summary: How to\u00a0Use Automation to Increase Test Coverage<\/a><\/span><br \/>\n<span style=\"color: #00b674;\"><a href=\"http:\/\/abstracta.us\/blog\/test-automation\/true-roi-test-automation\/\">The True ROI of Test Automation<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>What you need to know about test coverage Recently, I posted a blog about the ROI of test automation and its benefits. I mentioned that it increases test coverage, but do we really know what is test coverage? In this post, I\u2019ll go into more&#8230;<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[60],"tags":[65],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v14.0.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Automation Increases Test Coverage, But What is Test Coverage? | Abstracta<\/title>\n<meta name=\"description\" content=\"Recently, I posted a blog about the ROI of test automation. I mentioned that it increases test coverage, but do we really know what is test coverage?\" \/>\n<meta name=\"robots\" content=\"index, follow\" \/>\n<meta name=\"googlebot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta name=\"bingbot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Automation Increases Test Coverage, But What is Test Coverage? | Abstracta\" \/>\n<meta property=\"og:description\" content=\"Recently, I posted a blog about the ROI of test automation. I mentioned that it increases test coverage, but do we really know what is test coverage?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/AbstractaQA\/\" \/>\n<meta property=\"article:published_time\" content=\"2015-10-19T22:58:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-05T21:21:32+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/abstracta.us\/wp-content\/uploads\/2015\/10\/Test_coverage2-min.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"420\" \/>\n\t<meta property=\"og:image:height\" content=\"236\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@AbstractaUS\" \/>\n<meta name=\"twitter:site\" content=\"@AbstractaUS\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/abstracta.us\/blog\/#website\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\",\"description\":\"AI-powered quality engineering\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/abstracta.us\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"http:\/\/www.abstracta.us\/wp-content\/uploads\/2015\/10\/Test-Coverage-sweeper-211x300.jpg\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/#webpage\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/\",\"name\":\"Automation Increases Test Coverage, But What is Test Coverage? | Abstracta\",\"isPartOf\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/#primaryimage\"},\"datePublished\":\"2015-10-19T22:58:42+00:00\",\"dateModified\":\"2025-05-05T21:21:32+00:00\",\"author\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\"},\"description\":\"Recently, I posted a blog about the ROI of test automation. I mentioned that it increases test coverage, but do we really know what is test coverage?\",\"breadcrumb\":{\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/\",\"name\":\"Test Automation\"}},{\"@type\":\"ListItem\",\"position\":3,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/\",\"url\":\"https:\/\/abstracta.us\/blog\/test-automation\/say-automation-increases-test-coverage-test-coverage-exactly\/\",\"name\":\"They Say Automation Increases Test Coverage, but What Is Test Coverage Exactly?\"}}]},{\"@type\":[\"Person\"],\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\",\"name\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/abstracta.us\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/222e8b1136482564fe25acc4de2b9b7a?s=96&d=blank&r=g\",\"caption\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\"},\"description\":\"Co-Chief Executive Officer at Abstracta\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3591"}],"collection":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/comments?post=3591"}],"version-history":[{"count":18,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3591\/revisions"}],"predecessor-version":[{"id":17531,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/3591\/revisions\/17531"}],"wp:attachment":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/media?parent=3591"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/categories?post=3591"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/tags?post=3591"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}