{"id":1634,"date":"2015-05-24T22:44:02","date_gmt":"2015-05-24T22:44:02","guid":{"rendered":"http:\/\/www.abstracta.us\/?p=1634"},"modified":"2025-05-05T21:21:43","modified_gmt":"2025-05-05T21:21:43","slug":"the-tool-or-the-tester","status":"publish","type":"post","link":"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/","title":{"rendered":"What\u2019s More Important: The Performance Testing Tool or the Tester?"},"content":{"rendered":"<p><!-- Go to www.addthis.com\/dashboard to customize your tools --><script src=\"\/\/s7.addthis.com\/js\/300\/addthis_widget.js#pubid=ra-58d80a50fc4f926d\" type=\"text\/javascript\"><\/script><\/p>\n<h1><span style=\"font-weight: 400; color: #333333;\">What should we be more cautious about choosing: the tool we use or the person to run performance tests?<\/span><\/h1>\n<p><span style=\"font-weight: 400; color: #333333;\"><strong>Does the tool or the tester matter more for a good performance test?<\/strong> An aspect we always stress in performance testing is that the tools are circumstantial (at least to a certain degree).<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">What does this mean?<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">It&#8217;s true that different tools enable us to test different functions producing varying results. Anyone experienced in performance tests will be knowledgeable about the concepts, the types of performance tests,<\/span><span style=\"font-weight: 400; color: #333333;\">\u00a0the communications protocol, and monitoring. Therefore, he or she will be capable of managing any kind of tool, with more or less difficulty, because, despite the differences in scripting languages and the way in which things are done between one tool and the other, the base will always be the same.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\"><a href=\"http:\/\/scott-barber.blogspot.com\/\">Scott Barber<\/a>, one of the\u00a0greatest gurus in performance testing once said, <em>&#8220;Enterprise grade load generation tools are designed to look easy in sales demos. Don\u2019t be fooled.&#8221;<\/em><\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">So, what is more important&#8230; <strong>the tool\u00a0or the tester<\/strong>?<\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_Tester_is_More_Important_Than_the_Tool\"><\/span><strong><span style=\"color: #00b674;\">The Tester is More Important Than the Tool<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400; color: #333333;\">At Abstracta, we\u00a0always say this, but we could actually show how this is true due to one of our last consultancy projects. I had not handled the tool they used until then but I had the support of other team members whose help, in the end, I didn\u2019t need. Upon reviewing the scripts, I started to figure out how to use this new tool by doing a mapping of concepts between it and the tools I am already familiar with. In coordination with the team in charge of performing the tests, we could make all the scripts function, by simply applying the concepts that are already part of a proven method and by using best practices.<\/span><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Best_Practices\"><\/span><strong><span style=\"color: #333399;\">Best Practices<\/span><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<h4>Scripting\/Automation<\/h4>\n<ul>\n<li><span style=\"font-weight: 400; color: #333333;\">Verify that think times are not included in the response time measures.<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Parameterize tests, and verify that correlations function correctly.<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Test the script with different data, users, etc., so as to guarantee that the parameterization is working correctly (even controlling how data is impacted on the database).<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Perform validation for each step so as to have enough information, when executing the tests, to analyze which users had flaws, and in which step of which test case, and when using which data. That information will then be used to verify that the script is working and that the application functions with that data. If everything works, then the error occurred exclusively in the concurrence.<\/span><\/li>\n<\/ul>\n<h4>Execution<\/h4>\n<ul>\n<li><span style=\"font-weight: 400; color: #333333;\">Verify that the load generators are not saturated. If this happens, a load different from the one programmed will be generated, and the response times will not be measured properly.<\/span><\/li>\n<li><span style=\"font-weight: 400; color: #333333;\">Analyzing the results in the load generation tool is not enough. That is just half of the task. The most interesting part is obtained by cross-referencing this with the behavior of each component in the infrastructure (observing the monitored indicators, such as CPU, memory, network, etc., and with the help of specific tools to detect bottlenecks and opportunities for improvement, such as profiles, tracing of database, traffic analyzers, analyzing logs,<\/span><span style=\"color: #333333;\">\u00a0etc.)<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400; color: #333333;\">We could generally conclude that, nowadays, we use tools for programming, but it\u2019s necessary to learn good programming practices including good practices with each specific tool. The same goes for automated functional testing tools, performance testing tools, or monitoring tools, for example.<\/span><\/p>\n<p><span style=\"font-weight: 400; color: #333333;\">What is your experience regarding different testing tools?\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h2><span class=\"ez-toc-section\" id=\"Recommended_for_You\"><\/span><strong>Recommended for You<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><a href=\"http:\/\/abstracta.us\/blog\/performance-testing\/introduction-taurus-alternative-jmeter\/\">Introduction to Taurus: An Alternative to JMeter<\/a><br \/>\n<a href=\"http:\/\/abstracta.us\/blog\/performance-testing\/gatling-tool-review-performance-tests-written-scala\/\">Gatling Tool Review for Performance Tests (Written in Scala)<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>What should we be more cautious about choosing: the tool we use or the person to run performance tests? Does the tool or the tester matter more for a good performance test? An aspect we always stress in performance testing is that the tools are&#8230;<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[32,61],"tags":[37,127],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v14.0.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>What&#039;s More Important: The Performance Testing Tool or The Tester? | Abstracta<\/title>\n<meta name=\"description\" content=\"What should we be more cautious about choosing: the tool or the tester when it comes to running performance tests? You might be surprised!\" \/>\n<meta name=\"robots\" content=\"index, follow\" \/>\n<meta name=\"googlebot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta name=\"bingbot\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What&#039;s More Important: The Performance Testing Tool or The Tester? | Abstracta\" \/>\n<meta property=\"og:description\" content=\"What should we be more cautious about choosing: the tool or the tester when it comes to running performance tests? You might be surprised!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/AbstractaQA\/\" \/>\n<meta property=\"article:published_time\" content=\"2015-05-24T22:44:02+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-05T21:21:43+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/abstracta.us\/wp-content\/uploads\/2015\/05\/Tool_or_tester-min.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"420\" \/>\n\t<meta property=\"og:image:height\" content=\"236\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@AbstractaUS\" \/>\n<meta name=\"twitter:site\" content=\"@AbstractaUS\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/abstracta.us\/blog\/#website\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Blog about AI-powered quality engineering for teams building complex software | Abstracta\",\"description\":\"AI-powered quality engineering\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/abstracta.us\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/#webpage\",\"url\":\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/\",\"name\":\"What's More Important: The Performance Testing Tool or The Tester? | Abstracta\",\"isPartOf\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#website\"},\"datePublished\":\"2015-05-24T22:44:02+00:00\",\"dateModified\":\"2025-05-05T21:21:43+00:00\",\"author\":{\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\"},\"description\":\"What should we be more cautious about choosing: the tool or the tester when it comes to running performance tests? You might be surprised!\",\"breadcrumb\":{\"@id\":\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/\",\"url\":\"https:\/\/abstracta.us\/blog\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/performance-testing\/\",\"url\":\"https:\/\/abstracta.us\/blog\/performance-testing\/\",\"name\":\"Performance Testing\"}},{\"@type\":\"ListItem\",\"position\":3,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/\",\"url\":\"https:\/\/abstracta.us\/blog\/performance-testing\/the-tool-or-the-tester\/\",\"name\":\"What\\u2019s More Important: The Performance Testing Tool or the Tester?\"}}]},{\"@type\":[\"Person\"],\"@id\":\"https:\/\/abstracta.us\/blog\/#\/schema\/person\/78cd0dcae50ce820b25e86d3330e9762\",\"name\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/abstracta.us\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/222e8b1136482564fe25acc4de2b9b7a?s=96&d=blank&r=g\",\"caption\":\"Sof\\u00eda Palamarchuk, Co-CEO at Abstracta\"},\"description\":\"Co-Chief Executive Officer at Abstracta\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/1634"}],"collection":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/comments?post=1634"}],"version-history":[{"count":14,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/1634\/revisions"}],"predecessor-version":[{"id":11089,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/posts\/1634\/revisions\/11089"}],"wp:attachment":[{"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/media?parent=1634"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/categories?post=1634"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/abstracta.us\/blog\/wp-json\/wp\/v2\/tags?post=1634"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}