Cross-Browser-Testing with Jest & Selenium on Browserstack

Jest, Selenium, Browserstack?

First of all, we would like to show which functions the 3 tools perform in cross-browser testing.

Jest is the most popular testing framework and can theoretically be combined with Selenium. It fulfills three main functions:

1. It structures test suites and tests.
2. It allows us to write tests using convenient comparison functions that produce human-readable errors.
3. It allows execution and coverage reporting of test suites and tests.

Selenium addresses the W3C standardized WebDriver API to programmatically control a browser (click on a button, read the contents of an input field or paragraph, etc.). For example, Selenium can open google.com in Firefox, navigate to the search box, and type "esveo".

Browserstack provides a platform to use different browsers on different devices (for example, Firefox on Windows, Safari on macOS, or Chrome on Android). This ultimately allows running the same tests on different infrastructures ("Cross-Browser-Testing"). For this, Browserstack uses its real devices and opens our website. To enable that, the website must either be publicly available or we connect Browserstack with a tunnel to our localhost.

By the end of the article you should understand how it is possible to define test suites and tests with Jest, have these tests run automatically with Selenium in a real browser and how to connect Browserstack with Selenium to provide different operating systems and browser versions.

Connecting Selenium und Jest

First we set up the Jest configuration and environment variables. The environment variables allow us to securely store our Browserstack name and password later without storing them in the source code.

yarn add -D dotenv jest selenium-webdriver chromedriver \
  @types/jest @types/selenium-webdriver @types/chromedriver

# .env
BROWSERSTACK_USER=exampleUser
BROWSERSTACK_PASSWORD=examplePassword

Caution: The .env file should not be uploaded into the git repository, it should be gitignored. Each team member must then maintain their credentials on their machine.

After the environment variables, we define the configuration of Jest:

// jest.config.ts
import { Config } from "@jest/types";

const config: Config.InitialOptions = {
  /**
   * maxWorkers can be used to limit the number of parallel
   * tests. The number of allowed parallel tests
   * are dependent on the purchased
   * Browserstack license.
   **/
  maxWorkers: 1,

  /**
   * The test timeout can be increased,
   * to give the service more time to run tests,
   * before they are aborted. This may be necessary,
   * because the tests may take longer to run
   * due to the remote connection to the Browserstack device.
   **/
  testTimeout: 30000,
  /**
   * With "dotenv/config" in setupFiles we load the environment variables
   * from the .env file into the node process and can access them in the tests
   * with process.env.
   **/
  setupFiles: ["dotenv/config"],
};

export default config;

The following serves as an example component. We assume that it is callable at http://localhost:XXXX/test. This needs to be ensured by the build tool of your choice. The tests are completely independent of the framework, React is only chosen here to provide a compact example.

// Counter.tsx
import { useState } from "react";

export function Counter() {
  const [count, setCount] = useState(0);

  return (
    <>
      <p id="counter">{count}</p>
      <button id="button" onClick={() => setCount(count + 1)}>
        Click me
      </button>
      {count > 2 ? (
        <p id="counterIsBiggerThanTwoMessage">Count is bigger than 2</p>
      ) : null}
    </>
  );
}

A typical Jest test for this component using Selenium could be the following:

// Counter.test.ts;
/**
 * The Chromedriver must be imported,
 * so that Selenium can use Chrome.
 * Analogously you would need the "geckodriver" for Firefox.
 *
 * Later we remove the import from this file,
 * so that we can test on different platforms
 * without any adjustments to the tests.
 **/
import "chromedriver";

import { Builder, By, WebDriver, WebElement } from "selenium-webdriver";

/**
 * In this way we specify
 * the system that is used by Selenium.
 * Exactly in this way we can also set it later for
 * Browserstack. We can also make it dependent
 * on environment variables.
 **/
const capabilities = {
  os: "Windows",
  os_version: "10",
  browser: "chrome",
  browser_version: "latest",
  resolution: "1920x1080",
};

describe("Test Page", () => {
  /**
   * Here we bypass TypeScript a bit, so that we can later
   * more conveniently access the variables.
   **/
  let driver = null as any as WebDriver;
  let counter = null as any as WebElement;
  let counterUpButton = null as any as WebElement;

  /**
   * Before starting the first test, we let Selenium start Chrome
   * with the .build() function and the given specifications.
   */
  beforeAll(async () => {
    driver = await new Builder()
      .withCapabilities(capabilities)
      .forBrowser(capabilities.browser)
      .build();

    /**
     * We navigate to the page and let Selenium find the elements we need.
     * For the following line to work, at this point
     * a local WebServer must already be running.
     **/
    await driver.get("http://localhost:8080/#/test");
    counter = await driver.findElement(By.id("counter"));
    counterUpButton = await driver.findElement(By.id("button"));
  });

  /**
   * With our naming convention in describe() and test(), the test description
   * completes the page name (Here: "Test Page counter should increase when clicking the button.").
   **/
  test("counter should increase when clicking the button.", async () => {
    expect(await counter.getText()).toBe("0");
    await counterUpButton.click();
    expect(await counter.getText()).toBe("1");
  });

  test("should show a message when the counter is bigger than two.", async () => {
    expect(
      (await driver.findElements(By.id("counterIsBiggerThanTwoMessage"))).length
    ).toBe(0);
    await counterUpButton.click();
    await counterUpButton.click();
    expect(
      (await driver.findElements(By.id("counterIsBiggerThanTwoMessage"))).length
    ).toBe(1);
  });
});

With the describe(), test(), and expect() functions, we have built a typical Jest test infrastructure that compares whether elements have the values we expect. With Selenium, we access and interact with exactly these elements.

Separate application and test code

Until now, we have been accessing elements using the By function, more precisely the By.id() function of Selenium. An id is an attribute, which we typically address through HTML and CSS, i.e. application code. So we expect to be able to change this id if our application requires it, without having to change our tests. For example, we could make the following adjustment, creating an error in our test that until now has referred to the "button" id. And this even though the application still works as we want it to. This is of course a problem:

// Old code
// <button id="button" onClick={() => setCount(count + 1)}>
//   Click me
// </button>

<button id="bigButton" onClick={() => setCount(count + 1)}>
  Click me
</button>

We separate the application and test code by assigning a data-testid attribute to the elements and addressing them in the future. For this we change the code of our component and write a helper function for Selenium:

// Counter.tsx
<>
  <p data-testid="counter">{count}</p>
  <button data-testid="button" onClick={() => setCount(count + 1)}>
    Click me
  </button>
  {count > 2 ? (
    <p id="counterIsBiggerThanTwoMessage" data-testid="message">
      Count is bigger than 2
    </p>
  ) : null}
</>

// byTestId.ts
import { By } from "selenium-webdriver";

export function byTestId(id: string) {
  return By.css(`[data-testid=${id}]`);
}

// Counter.test.ts;
import { byTestId } from "/selenium-webdriver-extensions/byTestId";
// ...

// Example of old code: counter = await driver.findElement(By.id("counter"));
counter = await driver.findElement(byTestId("counter"));
counterUpButton = await driver.findElement(byTestId("button"));
// ...
expect((await driver.findElements(byTestId("message"))).length).toBe(0);
expect((await driver.findElements(byTestId("message"))).length).toBe(1);

Connect Selenium to Browserstack

To make this and all future tests Browserstack-ready, we offload the starting of the browser by Selenium (and the connection process to Browserstack) to a separate file that each test must call. To do this, we first customize the test as follows. The getTestUtils function is explained later:

// Counter.test.ts;
import "chromedriver";
import { Builder, WebDriver, WebElement } from "selenium-webdriver";
import { byTestId } from "/selenium-webdriver-extensions/byTestId";
import { getTestUtils } from "/test-infrastructure/getTestUtils";

/**
 * The testutils are included in each test file,
 * which will actually be executed in the real browser.
 * This way we can maintain unit tests and E2E tests in parallel.
 **/
const utils = getTestUtils();
beforeAll(async () => {
  await utils.setup();
});
afterAll(async () => {
  await utils.teardown();
});

describe("Test Page", () => {
  let counter = null as any as WebElement;
  let counterUpButton = null as any as WebElement;

  beforeAll(async () => {
    await utils.driver.get("http://localhost:8080/#/test");
    counter = await utils.driver.findElement(byTestId("counter"));
    counterUpButton = await utils.driver.findElement(byTestId("button"));
  });

  it("counter should increase when clicking the button.", async () => {
    expect(await counter.getText()).toBe("0");
    await counterUpButton.click();
    expect(await counter.getText()).toBe("1");
  });

  it("should show a message when the counter is bigger than two.", async () => {
    expect((await utils.driver.findElements(byTestId("message"))).length).toBe(
      0
    );
    await counterUpButton.click();
    await counterUpButton.click();
    expect((await utils.driver.findElements(byTestId("message"))).length).toBe(
      1
    );
  });
});

First of all, we now need to register at Browserstack and enter our Browserstack Automate Username and Access Key into the .env file. These can be found here and are not the Browserstack account email and password.

Next, we add the package that lets us connect localhost to Browserstack:

yarn add -D browserstack-local

import { Local } from "browserstack-local";
import { Builder, WebDriver } from "selenium-webdriver";

export function getTestUtils() {
  let driver = null as any as WebDriver;
  const browserstackLocalModule = new Local();

  /**
   * The setup function is called before each test.
   * It first connects localhost with Browserstack and
   * then lets Selenium start on Browserstack the
   * desired operating system-browser combination.
   **/
  async function setup() {
    /**
     * In contrast to the non-browserstack variant
     * additional capabilities must and should be defined.
     * To make the test on Browserstack identifiable,
     * it should be given a name.
     * Additionally you have to store your browserstack credentials,
     * as well as the attributes "browserstack.local" and
     * "browserstack.localIdentifier".
     **/
    const capabilities = {
      os: "Windows",
      os_version: "10",
      browser: "chrome",
      browser_version: "latest",
      resolution: "1920x1080",
      name: "Our first test",
      "browserstack.user": process.env.BROWSERSTACK_USER,
      "browserstack.key": process.env.BROWSERSTACK_PASSWORD,
      "browserstack.local": true,
      "browserstack.localIdentifier": "LocalConnection",
    };

    /**
     * This extension of the start function of the
     * browserstack-local module allows it to be used asynchronously,
     * so that await can be used to wait
     * for browserstack-local to start successfully.
     **/
    async function start() {
      return new Promise<void>((resolve, reject) => {
        browserstackLocalModule.start(
          {
            key: process.env.BROWSERSTACK_PASSWORD,
            localIdentifier: "LocalConnection",
          },
          (error) => {
            if (error) {
              reject(error);
            }
            resolve();
          }
        );
      });
    }
    await start();

    driver = await new Builder()
      // The Browserstack server
      .usingServer("https://hub-cloud.browserstack.com/wd/hub")
      .withCapabilities(capabilities)
      .forBrowser(capabilities.browser)
      .build();

    /**
     * This allows the tests
     * to call getTestUtils(), and
     * use the Selenium web driver.
     **/
    utils.driver = driver;
  }

  async function teardown() {
    await utils.driver.quit();
    // See start() function
    async function stop() {
      return new Promise<void>((resolve, reject) => {
        browserstackLocalModule.stop(() => {
          resolve();
        });
      });
    }
    await stop();
  }

  const utils = {
    setup,
    teardown,
    driver,
  };

  return utils;
}

All browser stack capabilities can be found here: https://www.browserstack.com/automate/capabilities

So the following code section must be added to each test to start, stop and use the Selenium web driver.

// Counter.test.ts

const utils = getTestUtils();
beforeAll(async () => {
  await utils.setup();
});
afterAll(async () => {
  await utils.teardown();
});

Afterwards, the dev server must be started and our Jest tests will be run on Browserstack from now on:

yarn jest Counter

Conclusion

At the time of writing this article, Browserstack in general or the browserstack-local module for Node.js generated errors regularly and without any discernible pattern. The connection could not be established approximately every 24 test runs, making it difficult to integrate Browserstack into a CI environment. Nevertheless, the tests provide us with great added value because we can test all interfaces in an automated way without having to work through all the UIs ourselves first.

Even though Jest is not the default solution for Selenium testing on Browserstack, this article has shown you how to easily enable it. Jest can also be used for your unit tests in addition to Selenium tests, and is extensible through the Jest ecosystem for example by Test Reporters.