mabl gives you great flexibility and control over when and where to run tests. From running a single test to validate it passes to running hundreds of regression tests in parallel across browsers, the choice is yours. You can easily implement a continuous testing strategy by running tests manually, on a schedule, and on a deployment event in your CI/CD pipeline so that every environment is covered with end-to-end tests.
Continuous testing with mabl
Here are some common use cases to help you better understand when and how to run tests:
- Replay test changes with the mabl Trainer on your local machine to validate everything works as expected.
- Trigger an ad-hoc test run in the mabl cloud against your staging environment to troubleshoot things with all available diagnostics data.
- Use the mabl CLI to run multiple tests in headless mode against a local development branch to validate that your feature changes didn’t break something.
- Run test changes saved to a mabl branch to test code that hasn’t yet been merged to a master or release branch
- Use the self-hosted mabl Runner to run mabl tests in headless mode inside your CI/CD environment to perform end-to-end tests on every code change, fast (available as private beta; request access).
- Trigger smoke and regression test runs in the mabl cloud on every deployment to staging and production environments in your CI/CD pipeline
- Test every code commit to a pull request (PRs) and see the test results directly inside the GitHub or BitBucket PR.
- Schedule plans to run all regression tests overnight against your staging environment across multiple browsers, including Internet Explorer 11 and Safari.
- Schedule a plan with key user journeys to monitor your production environment every few hours and get an alert when there is an issue with the user experience
You can run mabl tests in the following ways against any of your public or private test environments:
- Local - tests run on your own machine using the mabl Trainer or the mabl Runner, which is part of the mabl CLI
- Self-hosted - tests run inside your own CI/CD containers (servers) using the mabl Runner (available as private beta; request access)
- mabl cloud - tests run inside mabl’s highly secure and scalable infrastructure without limit on parallel runs
mabl test execution environments
As you create tests with the mabl Trainer, you can quickly replay the trained steps on your computer to validate that everything works as expected before saving your changes. Once the test is created, you can head over to the test details page in the mabl app to manually trigger an ad-hoc test run in the mabl cloud using the Run test button.
To run multiple tests as a group you should add them to a plan or tag them with a label. You would use test labels to do feature-based testing by running a small number of self-contained tests. On the other hand, plans give you the ability to specify test executions (e.g. parallel vs sequential, setup vs teardown), schedule runs, pass variables between tests, and more. You can also tag plans with labels to trigger multiple plan runs on a deployment event, for example.
You can use plans as a grouping mechanism to run tests in headless mode locally or in your CI/CD environment. This can be achieved using the
--from-plan option of the
mabl tests run-alpha command in the mabl CLI. However, plan features such as parallel runs, stages, and variable sharing between tests are not supported in the mabl Runner at the moment.
Here is how the different mabl test execution environments compare:
Feedback speed: single test
Feedback speed: multiple tests
DIY (bash scripts)
DIY (CI Tool)
Results reporting in mabl
Test diagnostics data
Plan runs support (stages, etc.)
Only as a grouping mechanism
Only as a grouping mechanism
Needs secure tunnel (mabl Link)
Counts towards monthly test runs
To be announced
*Running mabl tests inside your own CI/CD environment is available as a private beta and you can request access here.
Updated about a month ago