In mabl, you can run reliable end-to-end tests at every stage of your CI/CD pipeline. Continuous testing across CI/CD environments gives peace of mind to software developers, QA engineers, product managers, and other business stakeholders that issues are proactively resolved.
There are two ways to run mabl tests as part of your DevTestOps workflow — after a deployment to a test environment or even earlier in the development process, after a code build. This article provides an overview of those two options, including considerations around testing pull (merge) requests, and references for additional how-to resources.
Test on deployment
With deployment events, you can run mabl tests as soon as code is deployed to a hosting environment within your CI/CD pipeline. All mabl tests can run across environments by design, so you can easily run the same tests across multiple environments — from ephemeral preview environments for testing pull/merge requests to persistent staging environments, and even production.
Example of a CI/CD pipeline with 3 environments
Deployment events trigger tests to execute in the mabl cloud. Tests run in parallel and collect detailed diagnostics data, such as screenshots and network activity, helping you act on feedback faster.
Run mabl tests on deployment
To set up your mabl workspace for testing on deployment, organize your tests into one or more plans that you want to run. You can use plan labels to control which plans to run. At present, it is not possible to use test labels to run a group of tests for a deployment event in mabl.
After organizing your tests into plans, use one of the following options to create a mabl deployment event:
- Existing CI/CD integration - GitHub Actions, Jenkins, Bitbucket, among others.
- mabl CLI - integrate with any CI tools that can install and run Node.js packages.
- mabl API - integrate with any CI tools where you cannot use the mabl CLI or need more control
Mobile builds
If you are testing mobile builds on deployment, you can use the mabl CLI to programmatically manage your mobile builds. Learn more.
Test pull requests
Many software development teams run unit tests, static analysis, and various other code checks as part of their pull request (PR) review process. In some tools, this process is known as a merge request. PR checks are very valuable because they help catch issues early on in the development process. They also help streamline the code review process by providing PR reviewers with insight into the existence, or the lack thereof, of code quality regressions.
Historically, teams have avoided adding end-to-end UI tests as part of the PR checks because those tests were flaky, brittle, and very slow to run. However, this is no longer the case thanks to mabl’s dynamic waits, auto-healing, parallel execution, headless mode, and more. Performing end-to-end testing on the UI and/or API layers at the PR stages gives your team even higher confidence in approving the code changes and helps you accelerate your continuous delivery process.
mabl test results shown as a GitHub PR check
Run mabl tests as PR checks
You can start testing your PRs with mabl by using the integration with GitHub Checks or BitBucket Code Insights. Those integrations assume that your team deploys the pushed PR code to a test environment, usually ephemeral one, that you can test from the mabl cloud. You can establish a secure tunnel with mabl Link, if needed.
Use the mabl CI Runner
Alternatively, if a test environment for the PRs is not available or you simply do not want to run tests in the mabl cloud for every PR, you can configure the code repository to use the mabl CI runner and run tests in headless mode during the build process. With the mabl CI Runner, you can shift-left and get feedback from end-to-end tests earlier in your development lifecycle without having to deploy your application to a test environment.
The primary distinction with the mabl Runner is that these tests run inside of your CI/CD infrastructure. The tests and the application under test both run inside the CI container or virtual machine (VM).
Unlike cloud runs, CI runs don’t support parallel execution. They are best suited for quick pass/fail validation on a small set of tests before deployment to a shared environment. To learn more about the priorities and tradeoffs between different execution environments, check out the article on optimizing test execution with local, CI, and cloud runs.