Test execution overview

Mabl gives you great flexibility and control over when and where to run tests. From running a single test locally to validate it passes to running hundreds of regression tests in parallel across browsers, the choice is yours. You can also implement a continuous testing strategy by running plans manually, on a schedule, and on a deployment event in your CI/CD pipeline so that every environment is covered with end-to-end tests.

Here are some common use cases to help you better understand when to run tests:

1. Run individual tests for quick feedback and validation
2. Run multiple tests in a plan run or deployment event to validate that a feature works as expected before introducing new code into your application
3. Integrate tests into your CI/CD workflow to catch issues as early in the development process as possible
4. Set up smoke and regression tests to run on a schedule to ensure everything continues to work as expected
5. Run tests from the mabl CLI inside your local environment to validate your changes before adding them to your application.

You can run mabl tests locally, in the cloud, or in a self-hosted CI/CD environment against any of your public or private test environments.

Local runs

Local runs execute locally to your machine and do not count towards your monthly quota of allotted tests.

👍

Quick feedback

Local test runs work best as a diagnostic tool to get quick feedback on a single test.

Cloud runs

Tests executed in the cloud run on mabl’s highly secure and scalable infrastructure. You can run individual tests in the cloud or run tests as part of a plan.

Plan runs

Plan runs give you the ability to configure even more settings for test execution, including:

• parallel or sequential runs
• stages (for example: setup - test - teardown)
• scheduled runs
• sharing variables
• triggering tests via API
• integrating into a CI/CD workflow

Concurrency limits

The following table shows the maximum number of tests that can run in parallel at a given time for different types of tests:

If you trigger more than the maximum parallel test run limit, the remaining tests have an on hold status until the other test runs finish. If you need higher concurrency, reach out to the mabl Support Team.

Maximum run times

The maximum run times for cloud runs are as follows:

• Browser tests: six hours
• API tests: 30 minutes

📘

Browser tests in the cloud

Keep in mind the following for browser tests run in the cloud:

• Every cloud run uses a new instance, so mabl doesn’t cache your test data, including user sessions, between runs.
• The timezone in cloud runs is UTC.

Self-hosted - CI Runner

Tests run inside your own CI/CD containers (servers) using the mabl CI Runner.

Comparison of test execution environments

A full comparison of the different mabl test execution environments appears in the following table: