Mabl gives you great flexibility and control over when and where to run tests. From running a single test locally to validate it passes to running hundreds of regression tests in parallel across browsers, the choice is yours. You can also implement a continuous testing strategy by running plans manually, on a schedule, and on a deployment event in your CI/CD pipeline so that every environment is covered with end-to-end tests.
Here are some common use cases to help you better understand when to run tests:
- Run individual tests for quick feedback and validation
- Run multiple tests in a plan run or deployment event to validate that a feature works as expected before introducing new code into your application
- Integrate tests into your CI/CD workflow to catch issues as early in the development process as possible
- Set up smoke and regression tests to run on a schedule to ensure everything continues to work as expected
- Run tests from the mabl CLI inside your local environment to validate your changes before adding them to your application.
You can run mabl tests locally, in the cloud, or in a self-hosted CI/CD environment against any of your public or private test environments.
Local runs execute locally to your machine and do not count towards your monthly quota of allotted tests.
Local test runs work best as a diagnostic tool to get quick feedback on a single test.
Plan runs give you the ability to configure even more settings for test execution, including:
- parallel or sequential runs
- stages (for example: setup - test - teardown)
- scheduled runs
- sharing variables
- triggering tests via API
- integrating into a CI/CD workflow
The following table shows the maximum number of tests that can run in parallel at a given time for different types of tests:
If you trigger more than the maximum parallel test run limit, the remaining tests have an on hold status until the other test runs finish. If you need higher concurrency, reach out to the mabl Support Team.
The maximum run times for cloud runs are as follows:
- Browser tests: six hours
- API tests: 30 minutes
Browser tests in the cloud
Keep in mind the following for browser tests run in the cloud:
- Every cloud run uses a new instance, so mabl doesn’t cache your test data, including user sessions, between runs.
- The timezone in cloud runs is UTC.
Tests run inside your own CI/CD containers (servers) using the mabl CI Runner.
A full comparison of the different mabl test execution environments appears in the following table:
|Local||CI Runner (Self-hosted in CI/CD)||mabl cloud|
|Feedback speed: single test||Fastest||Faster||Fast|
|Feedback speed: multiple tests||Fast||Faster||Fastest|
|Scheduling||DIY (bash scripts)||DIY (CI Tool)||Yes|
|Results reporting in mabl||No||No||Yes|
|Test diagnostics data (for example screenshots, HAR logs, console logs)||Pass/Fail only||Pass/Fail only||Yes|
|Multibrowser support||Chrome only||Chrome only||Yes|
|Plan runs support||Only as a grouping mechanism||Only as a grouping mechanism||Yes|
|Needs secure tunnel (mabl Link)||No||No||Yes|
|Counts towards monthly test runs||No||No||Yes|
|Test runtime limit||No||No||Browser tests: 6hr|
API tests: 30 min
Updated 8 days ago