mabl gives you great flexibility and control over when and where to run tests. From running a single test to validate it passes to running hundreds of regression tests in parallel across browsers, the choice is yours. You can easily implement a continuous testing strategy by running tests manually, on a schedule, and on a deployment event in your CI/CD pipeline so that every environment is covered with end-to-end tests.
Here are some common use cases to help you better understand when and how to run tests:
- Replay test changes with the mabl Trainer on your local machine to validate everything works as expected.
- Trigger an ad-hoc test run in the mabl cloud against your staging environment to troubleshoot things with all available diagnostics data.
- Use the mabl CLI to run multiple tests in headless mode against a local development branch to validate that your feature changes didn’t break something.
- Run mabl end-to-end tests in headless mode as part of your JavaScript testing framework to catch issues as early in the development process as possible (available as private beta; request access)
- Run test changes saved to a mabl branch to test code that hasn’t yet been merged to a master or release branch
- Use the mabl Runner to run mabl tests in headless mode inside your CI/CD environment to perform end-to-end tests on every code change, fast (available as private beta; request access).
- Trigger smoke and regression test runs in the mabl cloud on every deployment to staging and production environments in your CI/CD pipeline
- Test every code commit to a pull request (PRs) and see the test results directly inside the GitHub or BitBucket PR.
- Schedule plans to run all regression tests overnight against your staging environment across multiple browsers, including Internet Explorer 11 and Safari.
- Schedule a plan with key user journeys to monitor your production environment every few hours and get an alert when there is an issue with the user experience
You can run mabl tests in the following ways against any of your public or private test environments:
- Local - tests run on your own machine using the mabl Trainer or the mabl Runner, which is part of the mabl CLI
- Self-hosted - tests run inside your own CI/CD containers (servers) using the mabl Runner (available as private beta; request access)
- mabl cloud - tests run inside mabl’s highly secure and scalable infrastructure without limit on parallel runs
Single test runs
As you create tests with the mabl Trainer, you can quickly replay the trained steps on your computer to validate that everything works as expected before saving your changes. Once the test is created, you can head over to the test details page in the mabl app to manually trigger an ad-hoc test run in the mabl cloud using the Run test button.
Multiple test runs
To run multiple tests as a group you should add them to a plan or tag them with a label. You would use test labels to do feature-based testing by running a small number of self-contained tests. On the other hand, plans give you the ability to specify test executions (e.g. parallel vs sequential, setup vs teardown), schedule runs, pass variables between tests, and more. You can also tag plans with labels to trigger multiple plan runs on a deployment event, for example.
Plan runs
You can use plans as a grouping mechanism to run tests in headless mode locally or in your CI/CD environment. This can be achieved using the --from-plan
option of the mabl tests run-alpha
command in the mabl CLI. However, plan features such as parallel runs, stages, and variable sharing between tests are not supported in the mabl Runner at the moment.
Here is how the different mabl test execution environments compare:
Feedback speed: single test
Fastest
Faster
Fast
Feedback speed: multiple tests
Fast
Faster
Fastest
Parallel runs
No
No
Yes (unlimited)
Scheduling
DIY (bash scripts)
DIY (CI Tool)
Yes
Results reporting in mabl
No
No
Yes
Test diagnostics data
Pass/Fail only
Pass/Fail only
Yes
Cross-browser support
Chrome only
Chrome only
Yes
Plan runs support (stages, etc.)
Only as a grouping mechanism
Only as a grouping mechanism
Yes
Insights (auto-heal, visual, JavaScript, etc.)
No
No
Yes
Needs secure tunnel (mabl Link)
No
No
Yes
Counts towards monthly test runs
No
To be announced
Yes
*Running mabl tests inside your own CI/CD environment is available as a private beta and you can request access here.
Updated 5 months ago