mabl gives you great flexibility and control over when and where to run tests. From running a single test to validate it passes to running hundreds of regression tests in parallel across browsers, the choice is yours. You can easily implement a continuous testing strategy by running tests manually, on a schedule, and on a deployment event in your CI/CD pipeline so that every environment is covered with end-to-end tests.

Use cases

Here are some common use cases to help you better understand when and how to run tests:

• Replay test changes with the mabl Trainer on your local machine to validate everything works as expected.
• Trigger an ad-hoc test run in the mabl cloud against your staging environment to troubleshoot things with all available diagnostics data.
• Use the mabl CLI to run multiple tests in headless mode against a local development branch to validate that your feature changes didn’t break something.
• Run mabl end-to-end tests in headless mode as part of your JavaScript testing framework to catch issues as early in the development process as possible.
• Run test changes saved to a mabl branch to test code that hasn’t yet been merged to a main or release branch.
• Use the mabl Runner to run mabl tests in headless mode inside your CI/CD environment to perform end-to-end tests on every code change.
• Trigger smoke and regression test runs in the mabl cloud on every deployment to staging and production environments in your CI/CD pipeline.
• Test every code commit to a pull request (PRs) and see the test results directly inside the GitHub or BitBucket PR.
• Schedule plans to run all regression tests overnight against your staging environment across multiple browsers, including Internet Explorer 11 and Safari.
• Schedule a plan with key user journeys to monitor your production environment every few hours and get an alert when there is an issue with the user experience.

Test execution

You can run mabl tests in the following ways against any of your public or private test environments:

• Local - tests run on your own machine using the mabl Trainer or the mabl Runner, which is part of the mabl CLI
• Self-hosted (CI Runner) - tests run inside your own CI/CD containers (servers) using the mabl CI Runner
• mabl cloud - tests run inside mabl’s highly secure and scalable infrastructure without limit on parallel runs

Single test runs

As you create tests with the mabl Trainer, you can quickly replay the trained steps on your computer to validate that everything works as expected before saving your changes. Once the test is created, you can head over to the test details page in the mabl app to manually trigger an ad-hoc test run in the mabl cloud using the Run test button.

Multiple test runs

To run multiple tests as a group you should add them to a plan or tag them with a label. You would use test labels to do feature-based testing by running a small number of self-contained tests. On the other hand, plans give you the ability to specify test executions (e.g. parallel vs sequential, setup vs teardown), schedule runs, pass variables between tests, and more. You can also tag plans with labels to trigger multiple plan runs on a deployment event, for example.

📘

Plan runs

You can use plans as a grouping mechanism to run tests in headless mode locally or in your CI/CD environment. This can be achieved using the --from-plan option of the mabl tests run-alpha command in the mabl CLI. However, plan features such as parallel runs, stages, and variable sharing between tests are not supported in the mabl Runner at the moment.

Here is how the different mabl test execution environments compare: