Unified Runner Overview

The Unified Runner is the next generation of the mabl testing service, built for more reliable and faster test execution. Reliability is achieved by unifying testing logic across the cloud, command-line interface, and Desktop app.

The Unified Runner replaces the legacy implementation for executing Chrome runs in the cloud. By default, ad hoc and plan runs will use the Unified Runner. For a limited period of time, existing accounts will have the option to use either the Unified Runner or the Legacy Runner for Chrome.

At this time, running tests both locally and in the cloud with the Unified Runner are available for Chrome only.

Local test runs can use the Unified Test Runner via the mabl CLI or the Desktop App.

Legacy Runner sunset

When the Unified Runner became generally available on March 21st, 2022, the Legacy Runner for Chrome entered the extended support phase. During this time, we encourage all existing customers to migrate any plans on the Legacy Runner to the Unified Runner, which can be accomplished by following these steps. Once this transition period ends, the Legacy Runner for Chrome will be formally retired.

Please note that trials and customers who joined after March 21st will not have access to the Legacy Runner for Chrome.

What is the extended support phase?
This means that the existing functionality of the Legacy Runner for Chrome will be maintained. However, new features may only be available on the Unified Runner. Bugs reported for the Legacy Runner will be addressed on a case-by-case basis.

What does it mean for the Legacy Runner to be retired?
Once the Legacy Runner is retired, all test runs on Chrome for supported test types will occur on the Unified Runner. The timeline for this change will be publicly announced in the coming months.

Are runs on the Unified runner still free?
During the beta period, runs through the Unified Runner did not count towards monthly run allocations. As part of the GA release, all runs on the Unified runner will be included in monthly run allocations starting on April 1st, 2022. You may notice a jump in the usage chart as a result of this change. Legacy test runs on Chrome will continue to count towards monthly run allocations.

Testing setup

The Unified Runner is on by default at both the plan and ad hoc levels.

This option can be found on the plan settings page. Any plans with this setting activated will run with the new test runner for supported browsers (Chrome only). Browsers not supported by the Unified Runner will fall back to the v1 test runner.

Unified Runner for plansUnified Runner for plans

Unified Runner for plans

The Unified Runner is also the default for ad hoc runs. If your team needs to use the Legacy Runner during the transition period, it can be enabled by simply flipping the Use the legacy test runner for Chrome toggle.

Unified Runner for ad hoc cloud runsUnified Runner for ad hoc cloud runs

Unified Runner for ad hoc cloud runs

📘

Default viewport size

If the width and height are not specified, the Unified Runner will default to a width of 1400 and a height of 1000 in the cloud. We recommend using a width and height that reflects the viewport size used when the test was created.

Improvements

In addition to the speed and reliability improvements, the following enhancements are available in the Unified Runner:

Test execution:

  • Ad-Hoc Cloud runs also capture Chrome step traces.
  • Unified runs in the cloud tailor timing to your application using Intelligent Wait.

Behavior changes

The following are known behavior changes in the Unified Runner.

Test creation:

  • Invalid CSS selectors that do not have a closing square bracket are no longer accepted. Existing tests need to be updated to properly format the CSS selector. For example, the Legacy Runner still allowed a CSS expression [aria-label="settings-portal-access". This is no longer accepted in the Unified Runner, and the expression must be updated to [aria-label="settings-portal-access"].
  • JavaScript syntax is enforced more strictly than the Legacy Runner. For example, variables must be explicitly defined.

Test execution:

  • When using XPath or CSS Selectors in a find elements step to enter text, the selector needs to find the input element in which you want to enter the text. In the Legacy Runner, if the element was not focused, it would go and try to type the text either way.

Limitations

The following limitations are present in the current version of the Unified Runner:

Browser support:

  • Only Chrome is supported at this time. More browsers to follow later.

Test creation:

  • Only tests and flows created with the Desktop App are currently supported.
  • Visual smoke tests are unsupported and will use the Legacy Runner.
  • Link crawler tests are unsupported and will use the Legacy Runner.
  • Visit homepage tests are unsupported and will use the Legacy Runner.

Learn more about identifying Unified test runs, intelligent wait, and migrating to the Unified Runner.


Did this page help you?