{"id":14763,"date":"2022-02-21T18:26:42","date_gmt":"2022-02-21T18:26:42","guid":{"rendered":"http:\/\/abstracta.us\/blog\/?p=14763"},"modified":"2025-05-05T21:25:21","modified_gmt":"2025-05-05T21:25:21","slug":"selenium-4-for-testers","status":"publish","type":"post","link":"https:\/\/abstracta.us\/blog\/software-testing\/selenium-4-for-testers\/","title":{"rendered":"What does Selenium 4 have in store for testers?"},"content":{"rendered":"\n

With the new version of the famous frontend automation tool made available, some novelties have surfaced and automation testers are more than keen to try them out. We will take a look at them to see exactly what has been introduced in this brand new version.<\/p>\n\n\n\n

<\/span>What is new?<\/strong><\/span><\/h2>\n\n\n\n

Complete switch to the W3C WebDriver Protocol from the legacy JSON Wire Protocol (JWP)<\/strong> that had been used in Selenium 3. In the new version JWP is no longer supported, but backward compatibility is in place, through Java bindings provided, so tests running in version 3 of Selenium won\u2019t break. That being said, under version 4 the communication between WebDriver and the browsers will be more stable, effective and consistent as they will share the same protocol, and no encoding or decoding will be needed to communicate through JWP. By the same token no more browser-specific tweaks are necessary to make it work in different browsers, as WebDriver is becoming W3C standardized. This also means that the Actions API has been revamped completely now conforming with the WebDriver Spec, though most of its modifications are internally used. In some cases the protocol shift may suppose a change in the name used in the capabilities such as the browser version which in JWP is described as version<\/em> would become browserVersion<\/em> in W3C.<\/p>\n\n\n\n

In short, switching over to W3C is a very beneficial change, mainly because all modern browsers use this protocol, so no more worrying about issues when porting tests from one browser to another, or adding complexity through JWP. As a result, less flakiness, more consistency and stable automated tests are expected to manifest during execution. .<\/p>\n\n\n\n

Enhanced feature to build a Selenium instance driver using the RemoteWebDriver builder method<\/strong> along with the adequate options for the intended browser (ChromeOptions, FirefoxOptions, etc). It is possible to include an address in case the intention is to connect the driver to a Grid server.<\/p>\n\n\n\n

Deprecations <\/strong>such as time signatures for waits and timeouts, now using Duration class, and deleted FindsBy* methods which in place have findElements with a By instance passed in as parameter.<\/p>\n\n\n\n

Relative locators:<\/strong> Now it is possible to find elements which are relative to the position of others. Depending on which side, from the element under question, we are looking for those other close elements, so it could be from the right, the left, above, below or simply near the one which has been already found.<\/p>\n\n\n\n

The relative location of an element is based on what is known as the client bounding rect<\/strong> which every element has and contains the information of the dimensions and position of the element, this way it knows exactly where it is located and where its edges are. Selenium 4 measures the distance from the edge of the element towards the intended direction, for example the bottom edge if we want to search in the below direction. The searched elements are returned according to the proximity to the relative element.<\/p>\n\n\n\n

What are its limitations? Due to the way it’s implemented it could happen that elements that should be found sometimes aren\u2019t. A good example of this is when we can find an element with a certain screen resolution, but then when we try to do so in another (maybe through running a remote Webdriver in another machine) the page layout might be different, which would cause the client bounding rects to not line up as previously, therefore not finding the elements.<\/p>\n\n\n\n

Register credentials easily through HasAuthentication<\/strong> class where we can store a set to use every time a user and password is needed or a set that matches against a predicate. These credentials are useful when we encounter basic or digest authentication. Prior to this method, in Selenium 3, the way to do it was through setting a login cookie in the session before accessing the site.<\/p>\n\n\n\n

Intercepting requests:<\/strong> sometimes the tests could seem flaky due to an unstable backend. Selenium 4 allows us to intercept the HTTP requests that the browser makes and even replace the resource requested with one from our own, such as an image for instance. What needs to be done is create a Network Interceptor with the driver and the route of the resource we want to replace.<\/p>\n\n\n\n

Chrome Debugging Protocol:<\/strong> many of the features of Selenium 4 are based on this protocol and it is used by chromium browsers for their debuggers. It is really insightful into the browser inner workings, therefore, this can be useful for testing. It can be accessed through the HasDevTools API and fortunately Selenium can speak multiple versions of this protocol so it can be used with different versions of a chromium-based browser. Selenium acts as a wrapper on the CDP API to send commands to the protocol and invoke the Chrome DevTools features.<\/p>\n\n\n\n

In this way we can try different scenarios such as: <\/p>\n\n\n\n

  1. Adding a network latency to the tests<\/li>
  2. Switching to another geolocation with latitude, longitude and accuracy parameters<\/li>
  3. Changing device screen dimensions<\/li>
  4. Blocking certain resources from loading such as images for faster tests<\/li>
  5. Print JavaScript errors appearing in the console<\/li><\/ol>\n\n\n\n

    <\/span>Selenium Grid<\/strong><\/span><\/h3>\n\n\n\n

    The Selenium Grid has been rebuilt to include different modes. While the classic Hub and Node is available, the server introduces two new setups, let\u2019s look at every single one of them <\/p>\n\n\n\n

    Hub and Node:<\/strong> Starting a hub server and then connecting the nodes with their setup<\/p>\n\n\n\n

    Standalone:<\/strong> the server listening on http:\/\/localhost:4444\/ rather than http:\/\/localhost:4444\/wd\/hub, and it is there that the Remote WebDrivers should point to. By default it will look for any drivers it can use by looking at the PATH.
    <\/p>\n\n\n\n

    Fully distributed:<\/strong> In this mode the hub has been revamped into multiple components, each in its own process. It has been created with the thought of modern infrastructure (Kubernetes). The processes are as follows and in the order they must be executed:<\/p>\n\n\n\n

    1. Event Bus \u2013 through here the node registers to the distributor<\/li>
    2. Session Map \u2013 stores the session ID and the node where the session is running, this info is sent to it by the Distributor<\/li>
    3. New Session Queue \u2013 Holds all the new session requests in FIFO order<\/li>
    4. Distributor \u2013 Watches all nodes to check for availability, if there is a matching request from the Session Queue it attaches it to the Node<\/li>
    5. Router \u2013 Gateway to the grid, connects clients to nodes. If it is a new session request it is added to the queue<\/li><\/ol>\n\n\n\n

      The new Grid now has integrated support for distributed tracing into it using OpenTelemetry<\/a>. It helps in revealing the internal states of the system using traces, metrics, and logs, this way it’s possible to know what is going on exactly. An open-source tool to monitor transactions, called Jaeger UI<\/a>, helps in visualizing these traces. The Grid is now easier to manage and the data can be queried from a GraphQL model to create custom visualizations in order to monitor it.<\/p>\n\n\n\n

      Last but not least it\u2019s worth mentioning that with the release of Selenium 4 the Selenium IDE has been brought back as a web-extensions based plugin available in Chrome and Firefox browsers, and with a brand new UI. Not only is it possible to export code for different languages, it also can create new commands, and plugins can be shipped as extensions, for instance Applitools for Selenium IDE for codeless visual testing. Additionally, The stability of the tests has been improved by including backup selectors, with different strategies, to fall back in case the main locators fail, and also conditions such as while or if can be included by the tester.<\/p>\n\n\n\n

      <\/span>From theory to practice<\/strong><\/span><\/h2>\n\n\n\n

      We decided to try these brand new features in a Page object model based framework, made from scratch using Selenium 4 for Java (JDK 14), TestNG, Maven, and WebdriverManager. For the purpose of this demo, we will be using the Abstracta opencart website<\/a>.<\/p>\n\n\n\n

      First of all, as mentioned before there is a new way to create a driver, by using the RemoteWebDriver builder, the difference comes in which capabilities is passed as parameter, so in this case we will be using ChromeOptions to indicate the Chrome browser.
      <\/p>\n\n\n\n

      driver = RemoteWebDriver.builder().oneOf(new ChromeOptions()).build();<\/code><\/pre>\n\n\n\n
      Additionally we could specify other details such as running the browser headless or indicating a grid address, but we won\u2019t be going into that right now.<\/p>\n\n\n\n
      <\/span>Relative Locators<\/strong><\/span><\/h3>\n\n\n\n
      To try this feature the plan was to head out to the all laptops section and find different products on the laptops list. <\/p>\n\n\n\n
      \"\"<\/figure><\/div>\n\n\n\n

      <\/p>\n\n\n\n
      The supplied product names are brought by a data provider, called by the test, in which names such as MacBook or Sony VAIO are provided. The first step is to find the first tag h2,<\/em> that appears above the laptops main image and highlight it, in the following way:<\/p>\n\n\n\n
      WebElement titleLaptops = driver.findElement(RelativeLocator.with(By.tagName(\"h2\")).above(laptops.getPageLoadedLocator()));\nWebDriverUtils.highlight(titleLaptops, driver);<\/code><\/pre>\n\n\n\n
      Let\u2019s not forget that the direction method can support a locator as well as an element already found.<\/p>\n\n\n\n
      <\/p>\n\n\n\n
      \"\"<\/figure>\n\n\n\n
      This step will always be successful as this is a simple one, however in the next ones let\u2019s introduce a different scenario. <\/p>\n\n\n\n

      What if we wanted to select the Add to cart<\/em> button for a specific product, can the relative locators help us with that task?  <\/p>\n\n\n\n
      \/\/ Find the right product\nWebElement desiredProduct = driver.findElement(By.linkText(productName));<\/code><\/pre>\n\n\n\n
       With the element found by its link text in the previous line (MacBook), what\u2019s next is finding the button in the different directions it could appear. As you have seen from the previous picture the button is located right below the product name, so we\u2019ll find every single product located below it and see what happens. <\/p>\n\n\n\n
      \/\/ Below\nList<WebElement> aux_list = driver.findElements(RelativeLocator.with(laptops.spanAddToCart).below(desiredProduct));<\/code><\/pre>\n\n\n\n
      <\/p>\n\n\n\n
      \"\"<\/figure><\/div>\n\n\n\n
      Not only could we find the desired button, but all others that stood right below the line where the product name is located, making for a total of 5 buttons. Given that they were found in order of proximity to the related element, the first one found was the one intended.<\/p>\n\n\n\n
      The next tries would be done finding the elements above, near (with an adjusted distance in pixels to 200 as the default is 100), left of, and right of the product name, so the quantity of elements given the different directions and browser modes were as follows: 
      <\/p>\n\n\n\n
      Chromedriver 97<\/strong><\/td>Graphic Browser<\/strong><\/td>Headless<\/strong><\/td><\/tr>
      Below<\/td>5 Shopping carts<\/td>0 Shopping carts<\/td><\/tr>
      Above<\/td>0 Shopping carts<\/td>5 Shopping carts<\/td><\/tr>
      Near<\/td>2 Shopping carts<\/td>0 Shopping carts<\/td><\/tr>
      Left<\/td>2 Shopping carts<\/td>5 Shopping carts<\/td><\/tr>
      Right<\/td>1 Shopping carts<\/td>0 Shopping carts<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
      Just as the feature\u2019s limitations were forewarned, when the DOM structure or the resolution changes, the elements may or may not be found in the same way again.<\/p>\n\n\n\n
      In the end Relative Locators are a fine addition to the Selenium functionalities as they are a useful approach when we want an element in a certain direction from another, however they might need a few improvements to deal with its limitations or with the example provided previously. In the latter case, there is always another solution, which is in turn finding the element through XPath navigation, from the specific name to the container element and then to the specific button related. The next selector displays this way to do it:<\/p>\n\n\n\n
      \"\/\/a[normalize-space()='MacBook']\/\/ancestor::div[@class='product-thumb']\/\/button[contains(@onclick,'cart')]\/span\"<\/code><\/pre>\n\n\n\n
      Relative locators examples<\/a><\/p>\n\n\n\n
      <\/span>Authentication<\/strong><\/span><\/h3>\n\n\n\n
      We can register credentials for a specific domain, or for common use with the following statements:<\/p>\n\n\n\n
      Predicate<URI> uriPredicate = uri -> uri.getHost().contains(\"the-internet.herokuapp.com\");\n((HasAuthentication) driver).register(uriPredicate,() -> new UsernameAndPassword(\"admin\",\"admin\"));<\/code><\/pre>\n\n\n\n
      In the latter case there is no need to register a predicate as the first parameter, with just the credential set it will do. No matter if it is Digest or Basic authentication, it will do the trick.<\/p>\n\n\n\n
      The website this time will be the-internet-herokuapp<\/a> website made to try different web functionalities.<\/p>\n\n\n\n
      <\/p>\n\n\n\n
      Authentication examples<\/a><\/p>\n\n\n\n
      <\/span>Network Interceptor<\/strong><\/span><\/h3>\n\n\n\n
      In this case, two different test cases were tried with the NetworkInterceptor <\/em>class, one for showing in the console every single request with its status code, in which a filter that uses an HttpHandler was passed in the Interceptor declaration, and this was the result:<\/p>\n\n\n\n
      <\/p>\n\n\n\n
      \"\"<\/figure>\n\n\n\n
      And another one in which a Routable is passed in the Interceptor, that checks for a specific image request and replaces it by one stored in the project, so that a different resource is loaded. For this demo, the images in the main slider will be replaced by a wallpaper.<\/p>\n\n\n\n
      Network Interceptor examples<\/a>
      <\/p>\n\n\n\n
      DevTools<\/strong><\/p>\n\n\n\n
      It is necessary to point out that not all browser drivers have the DevTools features as this is a Chromium exclusive, however Firefox seems to have it implemented as well, and needless to say it is, Opera and IE drivers are out of the question. Nonetheless, with the right driver it is possible to access a wide range of different of uses, of which we\u2019ve tried a few ones such as:<\/p>\n\n\n\n
      1. Block requested resources<\/li>
      2. Simulate Network conditions (different speeds or no connection at all)<\/li>
      3. Show DevTools console logs<\/li>
      4. Emulate Location Coordinates<\/li><\/ol>\n\n\n\n
        First of all in this specific test, what we need to do is have a DevTools<\/em> type variable which we will initialize for every test in this class by using the getDevTools() <\/em>and createSession<\/em> methods. Notice how in my case I had to cast my driver to ChromeDriver type to call these methods, as in my BaseTest I had it defined as a WebDriver. <\/p>\n\n\n\n
        private DevTools devTools = null;\n@BeforeMethod\npublic void setDevTools(){\n    devTools = ((ChromeDriver)driver).getDevTools();\n    devTools.createSession();\n}<\/code><\/pre>\n\n\n\n
        1- Block Requests<\/strong><\/p>\n\n\n\n
        In this case we are going to block all loaded JPG and PNG images as well as style-sheets, and see how it looks after doing a refresh.<\/p>\n\n\n\n
        devTools.send(Network.enable(Optional.empty(),Optional.empty(),Optional.empty()));\nList<String> blockedExtensions = ImmutableList.of(\"*.jpg\",\"*.png\",\"*.css\");\n\/\/ Block requested resources with a certain extension\ndevTools.send(Network.setBlockedURLs(blockedExtensions));<\/code><\/pre>\n\n\n\n
        <\/p>\n\n\n\n
        devTools.send(Network.emulateNetworkConditions(true,\n        0,\n        0,\n        0,\n        Optional.of(ConnectionType.CELLULAR3G)));<\/code><\/pre>\n\n\n\n
        In this case as we are setting offline mode, it is pointless to set the other parameters.<\/p>\n\n\n\n
        <\/p>\n\n\n\n
        devTools.send(Network.enable(Optional.empty(),Optional.empty(),Optional.empty()));\ndevTools.send(Network.emulateNetworkConditions(\n   false,\n   10000,\n   50000,\n   50000,\n    Optional.of(ConnectionType.CELLULAR3G)\n));\n\/\/ Timing for slow network\nlong startTimeSlow = System.currentTimeMillis();\ndriver.get(\"https:\/\/duckduckgo.com\/\");\nlong endTimeSlow = System.currentTimeMillis();\nSystem.out.println(\"Slow page loading under \" + (endTimeSlow - startTimeSlow) + \" milliseconds\");<\/code><\/pre>\n\n\n\n
        After running this code:  Slow page loading under 37788 milliseconds<\/em><\/strong><\/p>\n\n\n\n
        After commenting out the Network simulation:  Normal page loading under 903 milliseconds<\/em><\/strong><\/p>\n\n\n\n
        In the slow loading we can see that it took 38 seconds to load, while the normal one was barely a second<\/p>\n\n\n\n
        This is a useful way to try and see how a website reacts to a slow connection with different types such as 2G, 3G, Ethernet, etc, or no connection at all.<\/p>\n\n\n\n
        3- Console Logs<\/strong><\/p>\n\n\n\n
        Through the logs we can have more information about the current status of the website and to be able to see them displayed it is necessary to enable them first and then add a listener in which we can handle that data.Notice that we\u2019ll head to a \u201cthe-internet\u201d<\/em> page this time where there isn\u2019t much content to show. <\/p>\n\n\n\n
        \/\/ Enable logs\ndevTools.send(Log.enable());\n\/\/ Listener for logs\ndevTools.addListener(Log.entryAdded(), logEntry -> {\n    System.out.println(\"================\");\n    System.out.println(\"Log: \" + logEntry.toString());\n    System.out.println(\"Source: \" + logEntry.getSource());\n    System.out.println(\"Text: \" + logEntry.getText());\n    System.out.println(\"Timestamp: \" + logEntry.getTimestamp());\n    System.out.println(\"Level: \" + logEntry.getLevel());\n    System.out.println(\"URL: \" + logEntry.getUrl());\n});\n\/\/ Load page with console logs and wait\ndriver.get(\"http:\/\/the-internet.herokuapp.com\/portfolio\/\");<\/code><\/pre>\n\n\n\n
        \"\"<\/figure>\n\n\n\n
        4- Emulate Geolocation<\/strong><\/p>\n\n\n\n
        This time around we\u2019re going to simulate as if we were on another location changing the latitude and longitude as they are sent in the setGeolocationOverride<\/em> method. To do so we\u2019re going to load a website called https:\/\/where-am-i.org\/<\/em><\/a> which shows the location coordinates, and then after the emulation is set in place, reload the site to show the new ones.<\/p>\n\n\n\n
        driver.get(\"https:\/\/where-am-i.org\/\");\ndouble latitude = 64.128288, longitude = -21.827774;\n\/\/ Send in new location coordinates\ndevTools.send(Emulation.setGeolocationOverride(\n        Optional.of(latitude),\n        Optional.of(longitude),\n        Optional.of(1)));\nWebDriverUtils.ExplicitWaitElement(ExpectedConditions.visibilityOfElementLocated(marker),Constants.SHORT_TIMEOUT,driver);\nWebDriverUtils.MoveToElement(marker,driver);\nWebDriverUtils.highlight(driver.findElement(map),driver);\n\/\/ Reload page\ndriver.navigate().refresh();\nWebDriverUtils.ExplicitWaitElement(ExpectedConditions.visibilityOfElementLocated(marker),Constants.SHORT_TIMEOUT,driver);\nWebDriverUtils.MoveToElement(marker,driver);\nWebDriverUtils.highlight(driver.findElement(map),driver);\nWebDriverUtils.highlight(driver.findElement(lat),driver);\nWebDriverUtils.highlight(driver.findElement(longt),driver);\nAssert.assertEquals(driver.findElement(lat).getText(),String.valueOf(latitude));\nAssert.assertEquals(driver.findElement(longt).getText(),String.valueOf(longitude));<\/code><\/pre>\n\n\n\n
        <\/p>\n\n\n\n
        DevTools examples<\/a>
        <\/p>\n\n\n\n
        Summary<\/strong><\/p>\n\n\n\n
        In short, it is safe to say that Selenium 4 comes with a set of relevant features that can truly add up to the range of possibilities in the automation testing world, however while some of them might not be suitable for all circumstances, some are powerful enough to allow us to control browser settings to simulate certain conditions that wouldn\u2019t be possible before. From my perspective, the one that stands out is clearly the Chrome DevTools, as its potential to enhance tests makes Selenium mightier than it ever was.<\/p>\n\n\n\n
         Read more about it:<\/strong><\/p>\n\n\n\n
        \n
        Selenium IDE<\/a><\/blockquote>