Comparing the two protocols to beat a fun automation game

I was inspired recently by Federico Toledo to write a test script that can automate a simple clicking game, but with a focus on executing as fast as possible.

Test automation challenge: Can you prepare a test script to win the game 1to50 automatically as fast as possible https://t.co/H6mwYxeKko???

My solution with Ruby and Selenium took 4.9secs to win the game and I know it can be faster https://t.co/HadbaCzien pic.twitter.com/UrPDh2bfGX

— Federico Toledo (@fltoledo) April 11, 2020

This challenge inspired me to use my favorite Web Testing Framework, webdriverIO, which supports Chrome DevTools and WebDriver protocols out-of-the-box with minor setup.

So I decided to try and see which protocol executes faster.

Let’s Explain the Details First

In order to make your tests scripts execute actions against a website, your test framework has to use an interface which is actually capable of controlling the browser.

There are 2 very popular interfaces that allow this to happen. Selenium uses WebDriver (w3c protocol) which is today the most popular one, but there is also Chrome DevTools which is native to chromium browsers and has some benefits and disadvantages over its competitor.

In this post we are focusing on speed and spoiler alert: our winner is DevTools.

Here’s Why!

WebDriver sits in the middle between the test scripts and the browser and uses http traffic to communicate back and forth which can cause a small latency between the execution and the response from the browser.

On the other hand, Chrome DevTools allows direct control over the browser by using the Puppeteer library which is finally used by webdriverIO. This means direct execution and less latency in the results.

Let’s Do the Test

The idea of the game is to click the numbers from 1 to 50 as fast as possible, but only the next 25 are displayed. So if you click 1 then the 26th appears. 

I created a simple script using webdriverIO on standalone mode so that it runs fast without all the test framework setup.

const { remote } = require('webdriverio');
(async () => {
    const browser = await remote({
        logLevel: 'trace', capabilities: { browserName: 'chrome'}
    })
    await browser.url('https://zzzscore.com/1to50/en/?ts=1592666149743') // navigate
    for(i = 1; i<51; i++){
        var element = await browser.$('//div[text()="@id"]'.replace('@id',i))
        await element.click(); //click each number
    }
    var result = await browser.$('.level');
    console.log(`The result is ${await result.getText()}`);
    // await browser.deleteSession()
})().catch((e) => console.error(e))

By default, webdriverIO v6 will try to connect using devtools protocol unless an explicit WebDriver path is specified.

To run the script just type:

node <scriptname>.js

DevTools Result

The first thing you will notice when executing the script is this line of log: INFO webdriverio: Initiate new session using the devtools protocol which indicates the protocol used.

And you’ll get this result: The result is 0.85. Which is really fast, meaning at least 50 browser interactions occurred in less than a second.

WebDriver Result

In order to execute the same script on WebDriver protocol, we need to specify the hostname and path when creating the remote object. We also require a running WebDriver server. I recommend installing the webdriver-manager package which simplifies the process a lot.

Once the WebDriver is running (by default it runs on localhost:444/wd/hub), then just update the code.

...
 const browser = await remote({
        logLevel: 'trace', capabilities: { browserName: 'chrome'},
        hostname: 'localhost', path:'/wd/hub'
    })
...

Again you will notice that the WebDriver protocol is specified: INFO webdriverio: Initiate new session using the webdriver protocol.

And the result is: 3.801 which is impressive, but it’s four times slower than the first execution and puts WebDriver in second place.

Final Thoughts

We can easily conclude that the DevTools protocol executes scripts way faster which makes me consider it as a scripting and scraping tool. But there’s no free lunch, WebDriver is nowadays the most used interface and the one that has support for cross-browser testing, devices and even desktop apps.

I’d also like to mention the great capability of webdriverIO to support both protocols out of the box. It really matters a lot when a tool allows you the power of choice.

Credits for the protocols images to the webdriverIO website. This article was originally published here on Dev.to

---

Recommended for You

Top Test Automation Tools to Try in Your Next Project
Migrating to Open Source Testing Tools (Especially Now)