Use the “run mabl tests” GitHub Action to automatically trigger mabl tests as a part of your GitHub Actions workflow. With the “run mabl tests” GitHub Action, you can:
- Run mabl plans associated with a specific mabl environment and/or application as soon as you initiate a new deployment in GitHub
- View results of mabl tests directly within GitHub
- Customize subsequent actions in the pipeline based on the results of mabl tests
The Action triggers a mabl deployment event, which runs tests in the mabl cloud. To run tests on your CI runner instead, see Running tests inside a CI environment.
Setup
To set up the “Run mabl tests” GitHub Action, take the following steps:
- In the mabl app, create a “CI/CD integration” API key. API key access is limited to workspace owners.
- In GitHub, use GitHub Actions Secrets to store the mabl API key with the name
MABL_API_KEY. - Add a “run mabl tests” GitHub Action to your repo’s GitHub Actions workflow.
- Update the placeholder values in the Action with the parameters for the mabl tests you want to run. At a minimum, include the application ID and/or environment ID of the mabl plans you want to trigger.
For a complete list of parameters for running mabl tests, including browsers, branches, plan labels, and URL override, see Example workflow: Complete.
The Action automatically passes your commit hash, github.sha, to mabl as part of the deployment event, which can be used for automatic GitHub issue creation and/or and custom GitHub events.
Combine the mabl GitHub Action and mabl GitHub integration
Installing the mabl GitHub integration (app) allows for richer interactions, like results in Pull Requests. See GitHub integration setup for the full details.
Customize the Action
The example workflows trigger mabl tests when you push a commit or tag. Depending on what you want to accomplish, you may use a different event to trigger mabl tests, such as opening a pull request or setting tests to run on a schedule.
You can also control what subsequent actions are executed by your pipeline using test results statistics. For example, you can:
- Customize your pipeline to fail based on a specific threshold or percentage of passing tests instead of failing automatically on a single test failure.
- Configure your pipeline to automatically promote the build to the next environment in the event that all tests pass to eliminate the need to manually tag or trigger a push to development or production.
Run the GitHub Action
As soon as mabl receives a deployment event from your pipeline, it automatically triggers a mabl deployment event based on the specific parameters defined in the Action. The Action continually polls mabl until all triggered tests are completed.
Unless you configured a different behavior on failure, the pipeline stops if the Action returns a failure.
Plan-level auto-retries do not update the Action job status
If your plan has the “Automatically retry on failure” setting enabled, the Action reports the result of the initial plan run. If the initial run fails and the retry passes, the Action job remains failed in GitHub even though the plan ultimately passed in mabl. If you rely on retry behavior to determine pass/fail, check the plan run in mabl rather than the Action job status.
View Action logs
After a deployment has occurred, you can see the results of the mabl tests triggered by the Action directly in the GitHub Action logs:
- Click on the Actions tab and open the mabl GitHub Actions workflow to view Action logs.
- If you configured the mabl GitHub Action to run on a PR, you can also click on the Details button to view the Action logs.
Viewing output logs for the GitHub Action
GitHub Actions logs include helpful information about the mabl tests including a table with information about the: browser, status, test name, duration of the run, and a direct link back to the output in the mabl app.
Troubleshooting
Deployment succeeded but no tests ran
If the Action returns success quickly, for example within a minute, and you see a deployment event in mabl but no plan runs, it’s likely that no plans matched the deployment event. For mabl to trigger a plan in response to a deployment event, the plan must meet all of the following criteria:
- The plan’s filters - application, environment, and plan labels - match the parameters of the deployment event. Label matching is case-sensitive.
- The plan is toggled on and active.
- The plan is configured to accept a deployment trigger in the Triggers section of the plan settings.
- At least one of the tests in the plan is enabled.
For more details, see Deployment events.
401 Unauthorized errors
If the Action returns a 401 error, check the following:
- API key type: Confirm the API key in mabl is a “CI/CD integration” type. Other API key types, such as Editor or Deployment Trigger, will not authenticate the Action.
-
Secret name match: Confirm the GitHub secret name matches the variable name in your workflow YAML exactly. Secret names are case-sensitive. For example,
MABL_API_KEYandmabl_api_keyare different. - Secret scope: Confirm the secret is defined at the repository level, not only in a deleted environment or in a different repository.