Unified test runner overview

The Unified test runner is the next generation mabl testing service, built for more reliable and faster test execution. Reliability is achieved by unifying testing logic across the cloud, command-line interface, and Desktop app.

The Unified test runner will eventually replace the current implementation for executing tests in the cloud (v1). During the transition period, you will have the option to use either the current v1 runner or the new Unified runner.

At this time, the runner is available for only Chrome cloud executions. For local runs, the Unified test runner can also be accessed by using the mabl CLI or Desktop App.

📘

Beta Access for Trial Workspaces

The Unified test runner beta is currently available to all mabl customers. Changes may be made without notice until the runner becomes generally available. Trial users should reach out to their sales representative to express interest in this feature.

Testing Setup

The Unified test runner is currently available at both the plan and ad hoc levels.

This option can be found in the plan settings. On the Edit Plan page:

  1. Expand the Advanced section in the Browsers test settings section
  2. Use the toggle to enable the Unified test runner
  3. Save the changes to the plan

Now you are all set up to use the Unified runner! Any plans with this setting activated will run with the new test runner for supported browsers (Google Chrome only). Browsers not supported by the Unified test runner will fall back to the v1 test runner.

Unified test runner enabled for Google Chrome testsUnified test runner enabled for Google Chrome tests

Unified test runner enabled for Google Chrome tests

The Unified test runner can also be selected for ad hoc runs. This option can be found in the Ad hoc cloud run panel under the Use Unified Test Runner toggle.

Use the Unified test runner for ad hoc cloud runsUse the Unified test runner for ad hoc cloud runs

Use the Unified test runner for ad hoc cloud runs

📘

Default viewport size

If the width and height are not specified, the Unified runner will default to a width of 1400 and a height of 1000 in the cloud. We recommend using a width and height that reflects the viewport size used when the test was created.

Improvements

In addition to the speed and reliability improvements, the following enhancements are available in the Unified test runner:

  • Test Execution:
  • Ad-Hoc Cloud runs also capture Chrome step traces.

Behavior Changes

The following are known behavior changes in the Unified test runner.

  • Test creation:

  • Invalid CSS selectors that do not have a closing square bracket are no longer accepted. Existing tests need to be updated to properly format the CSS selector. For example, the v1 test runner still allowed a CSS expression [aria-label="settings-portal-access". This is no longer accepted in the Unified test runner, and the expression must be updated to [aria-label="settings-portal-access"].

  • Test Execution:

  • When using XPath or CSS Selectors in a find elements step to enter text, the selector needs to find the input element in which you want to enter the text. In the v1 test runner, if the element were not focused, it would go and try to type the text either way.

Limitations

The following limitations are present in the current version of the Unified test runner:

  • Browser support: only Chrome is supported at this time. More browsers to follow later.

  • Test creation:

    • Only tests and flows created with the mabl training extension or Desktop App are currently supported
      • Visual smoke tests are unsupported and will use the v1 test runner
      • Link crawler tests are unsupported and will use the v1 test runner
      • Visit homepage tests are unsupported and will use the v1 test runner
  • Test results:

    • Only basic element find activity logs are available
  • Results:

    • Tests run with the Unified test runner are not billed for during the beta period

Learn more about identifying Unified test runs and fine-tuning page interaction speed.


Did this page help you?