Test execution overview
mabl gives you great flexibility and control over when and where to run tests. From running a single test to validate it passes to running hundreds of regression tests in parallel across browsers, the choice is yours. You can easily implement a continuous testing strategy by running tests manually, on a schedule, and on a deployment event in your CI/CD pipeline so that every environment is covered with end-to-end tests.


Continuous testing with mabl
Use cases
Here are some common use cases to help you better understand when and how to run tests:
- Replay test changes with the mabl Trainer on your local machine to validate everything works as expected.
- Trigger an ad-hoc test run in the mabl cloud against your staging environment to troubleshoot things with all available diagnostics data.
- Use the mabl CLI to run multiple tests in headless mode against a local development branch to validate that your feature changes didn’t break something.
- Run mabl end-to-end tests in headless mode as part of your JavaScript testing framework to catch issues as early in the development process as possible.
- Run test changes saved to a mabl branch to test code that hasn’t yet been merged to a main or release branch.
- Use the mabl Runner to run mabl tests in headless mode inside your CI/CD environment to perform end-to-end tests on every code change.
- Trigger smoke and regression test runs in the mabl cloud on every deployment to staging and production environments in your CI/CD pipeline.
- Test every code commit to a pull request (PRs) and see the test results directly inside the GitHub or BitBucket PR.
- Schedule plans to run all regression tests overnight against your staging environment across multiple browsers, including Internet Explorer 11 and Safari.
- Schedule a plan with key user journeys to monitor your production environment every few hours and get an alert when there is an issue with the user experience.
Test execution
You can run mabl tests in the following ways against any of your public or private test environments:
- Local - tests run on your own machine using the mabl Trainer or the mabl Runner, which is part of the mabl CLI
- Self-hosted (CI Runner) - tests run inside your own CI/CD containers (servers) using the mabl CI Runner
- mabl cloud - tests run inside mabl’s highly secure and scalable infrastructure without limit on parallel runs


Single test runs
As you create tests with the mabl Trainer, you can quickly replay the trained steps on your computer to validate that everything works as expected before saving your changes. Once the test is created, you can head over to the test details page in the mabl app to manually trigger an ad-hoc test run in the mabl cloud using the Run test button.
Multiple test runs
To run multiple tests as a group you should add them to a plan or tag them with a label. You would use test labels to do feature-based testing by running a small number of self-contained tests. On the other hand, plans give you the ability to specify test executions (e.g. parallel vs sequential, setup vs teardown), schedule runs, pass variables between tests, and more. You can also tag plans with labels to trigger multiple plan runs on a deployment event, for example.
Plan runs
You can use plans as a grouping mechanism to run tests in headless mode locally or in your CI/CD environment. This can be achieved using the
--from-plan
option of themabl tests run-alpha
command in the mabl CLI. However, plan features such as parallel runs, stages, and variable sharing between tests are not supported in the mabl Runner at the moment.
Here is how the different mabl test execution environments compare:
Local | CI Runner (Self-hosted in CI/CD) | mabl cloud | |
---|---|---|---|
Feedback speed: single test | Fastest | Faster | Fast |
Feedback speed: multiple tests | Fast | Faster | Fastest |
Parallel runs | No | No | Yes (unlimited) |
Scheduling | DIY (bash scripts) | DIY (CI Tool) | Yes |
Results reporting in mabl | No | No | Yes |
Test diagnostics data | Pass/Fail only | Pass/Fail only | Yes |
Cross-browser support | Chrome only | Chrome only | Yes |
Plan runs support (stages, etc.) | Only as a grouping mechanism | Only as a grouping mechanism | Yes |
Insights (auto-heal, visual, JavaScript, etc.) | No | No | Yes |
Needs secure tunnel (mabl Link) | No | No | Yes |
Counts towards monthly test runs | No | No | Yes |
Test runtime limit | No | No | Chrome: 6hr |
Custom HTTP Header Support | Limited | Limited | Full |
Updated 22 days ago