Plan stages

Parallel vs. sequential runs; set-up and tear-down stages

Introduction to plan stages

In order to minimize run time and dependencies between tests, mabl recommends creating test that can be run in parallel whenever possible. However, we recognize that in some cases it is necessary to have more control over how tests are run within a plan. Plan stages can be used in those situations to model custom run behaviors. By grouping your tests into stages you will be able to control both the concurrency of the tests in each stage (parallel or sequential) as well as the order in which tests will run.

Properties of plan stages

A plan stage groups tests and adds controls that allow you to decide how those tests should be run. The two settings that can be changed in a plan stage are concurrency and always run. The two concurrency options are:

  • Parallel: All tests in this stage will start running at the same time
  • Sequential: tests in this stage will run one at a time in the order in which they are defined in the plan

When the always run setting is enabled, that stage will always be run, even if a tests in a previous stage failed. This property is useful for defining clean-up stages that will remove test data that was generated by an earlier stage.

Sequential execution

Plans with multiple browsers configured will execute sequential stages one browser at a time. The order of execution is not configurable.
tests with data tables in a sequential stage will also be executed one scenario at a time.

Ordering of plan stages

The order in which stages are defined in a plan determines the order in which each stage will run. A later stage will not begin running until all previous stages have completed. This behavior is useful for modeling standard set-up / test / clean-up behavior. For example, a hypothetical plan might be define three stages like the following:

1. Set-up
Concurrency: sequential
Tests:

  • Create user
  • Add vehicle

2. Vehicle tests
Concurrency: parallel
Tests:

  • Schedule car wash
  • Order new tires
  • Run diagnostics

3. Clean-up
Concurrency: sequential
Always Run: true
Tests:

  • Remove vehicle
  • Remove user

Creating a plan with stages

For new plans and plans that were created prior to the introduction of stages, the default behavior is to have a single parallel stage. These settings can be changed by editing the stage or adding new stages.

After clicking the edit button you can then specify the properties of the stage:

Once you have defined your stages and saved the plan, you will see your tests grouped by stage in the upper right of the plan page:

Behavior on failure

When a test fails, the following events will occur:

  1. If the test was part of a sequential stage, all tests in that stage that come after the failed test will be skipped (i.e. will not start running)
  2. All tests in subsequent non-always-run stages will immediately be skipped
  3. The first always-run stage (if one exists) will begin running

As an example, the following image shows what happened when a sequential test failed in the first of three stages. The third and fourth tests in the first stage were skipped, all tests in the second stage were skipped, and the tests in the third (always-run) stage were run:

Updated about a year ago


Plan stages


Parallel vs. sequential runs; set-up and tear-down stages

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.