Run change validation from the CI pipeline you already have. Your pipeline tells mabl when a pull request or deployed commit reaches a stage in your policy file, and mabl runs that stage's suites and impacted tests. Your pipeline then gets the result and passes or blocks the change.
There are two ways to set it up:
- With GitHub Actions, which we recommend if your repository is on GitHub. The mabl change validation GitHub Action does the work in a few lines of workflow.
- With any other CI system, such as Azure DevOps. Your pipeline calls the mabl API itself, so there is more to build and maintain.
Early access
Change validation is in early access for accounts with generative AI features. Pricing and packaging are subject to change.
To learn what mabl does at each stage, see How change validation works.
Before you start
You need:
- A mabl account with generative AI features, and your workspace ID. See mabl resource IDs.
- A policy file in your repository. See Set up a mabl policy file.
- A "Command Line Interface" or "Editor" API key made for your pipeline. Other API key types can't start change validation.
With GitHub Actions
The mabl change validation GitHub Action reads your policy file, tells mabl about each stage, and reports the result.
-
In your GitHub repository, store your mabl API key as an encrypted secret named
MABL_API_KEY. -
Add a workflow file to
.github/workflows, such asmabl-pull-request.yml, with this content and your workspace ID:name: mabl pull request on: pull_request: types: [opened, synchronize, reopened, edited] jobs: mabl: permissions: contents: read pull-requests: read uses: mablhq/change-validation-action/.github/workflows/pull-request.yml@v1 with: stage: pull_request workspace-id: '[workspace-id]' secrets: mabl-api-key: ${{ secrets.MABL_API_KEY }} -
Open a pull request. mabl validates the
pull_requeststage of your policy file, and the tests appear in mabl as a deployment event.
To block a merge on the result, add a second workflow for the merge queue. Run it after the job that deploys the queue's commit. It waits for mabl and publishes the result as a commit status. Require that status, mabl-merge-queue, in your branch rules:
name: mabl merge queue
on:
merge_group:
jobs:
mabl:
permissions:
contents: read
pull-requests: read
statuses: write
uses: mablhq/change-validation-action/.github/workflows/merge-queue.yml@v1
with:
stage: merge_queue
workspace-id: '[workspace-id]'
status-context: mabl-merge-queue
secrets:
mabl-api-key: ${{ secrets.MABL_API_KEY }}
GitHub lets a pull request into the merge queue only after its required checks pass on the pull request itself. mabl publishes mabl-merge-queue only once the pull request is in the queue, so each pull request also needs that status published as a success before it can enter. The action's repository has a placeholder workflow that publishes it.
The action can also keep a comment on the pull request with mabl's report, validate a commit after it deploys, and check your policy file on the pull requests that change it. For those workflows and every input, see the action's README.
With any other CI system
On another CI system, your pipeline makes the API calls that the action makes on GitHub. This path is manual: you write and maintain the steps below. If your repository is on GitHub, use the action instead.
For a pull request, a pipeline makes up to four calls:
-
A stage request, each time the pull request reaches a stage of your policy.
POST /changeValidation/events -
A status call, when you need the result.
POST /changeValidation/status -
A report call, if you want mabl's results on the pull request.
POST /changeValidation/report -
A merged event, when the pull request merges, if your policy merges mabl branches.
POST /changeValidation/events
The first stage request for a pull request starts a mabl agent for it. The agent handles every later request for that pull request, with the permissions of the API key that sent the first one.
The sections below walk through one stage of a pull request. For every field of each call, see the mabl API reference. For the rules that apply to merge queue entries, deployed commits, and merged pull requests, see Build your own client in the action's repository.
Ask mabl to validate a stage
Send a stage request each time a pull request reaches a stage, such as when it opens or gets a new commit. Skip pull requests from forks.
- Fetch the pull request's current head commit, base commit, branch name, and labels.
- Read
.mabl/policy.yamlat the pull request's base commit, not its head, so an author can't change the rules in their own pull request. If the read fails, retry it instead of sending the request. - Send the request to
POST https://api.mabl.com/changeValidation/events.
For example, suppose pull request #42 in the acme/shop repository on GitHub, from the branch add-gift-cards, has opened. This script builds the request for its pull_request stage from values your CI provides in environment variables, and sends it:
set -euo pipefail
POLICY_PATH=.mabl/policy.yaml
git cat-file -e "$BASE_SHA^{commit}"
if git cat-file -e "$BASE_SHA:$POLICY_PATH" 2>/dev/null; then
git show "$BASE_SHA:$POLICY_PATH" > policy.yaml
POLICY=$(jq -n --rawfile text policy.yaml --arg sha256 "$(sha256sum policy.yaml | cut -d ' ' -f 1)" \
'{found: true, text: $text, sha256: $sha256}')
else
POLICY='{"found": false}'
fi
jq -n \
--arg workspace "$WORKSPACE_ID" --arg repo "github.com/acme/shop" --argjson pr 42 \
--arg stage "pull_request" --arg head "$HEAD_SHA" --arg base "$BASE_SHA" --arg ref "add-gift-cards" \
--argjson at "$OCCURRED_AT" --argjson labels "$LABELS" --arg description "$DESCRIPTION" \
--arg path "$POLICY_PATH" --argjson policy "$POLICY" '{
workspace_id: $workspace,
reference: {type: "pull_request", id: "\($repo)#\($pr)"},
event: {kind: "stage.requested", stage: $stage, head_sha: $head, base_sha: $base,
head_ref: $ref, occurred_at: $at, labels: $labels},
change_description: $description,
related_artifacts: [{artifact_type: "inline", name: "validation_policy", artifact_data: {type: "json",
value: ({repository: $repo, pull_request: $pr, base_sha: $base, head_sha: $head,
head_ref: $ref, path: $path} + $policy)}}]
}' > stage-request.json
curl --request POST \
--url https://api.mabl.com/changeValidation/events \
--header 'accept: application/json' \
--user "key:$MABL_API_KEY" \
--header 'content-type: application/json' \
--data @stage-request.json
A few values in the request deserve care:
-
reference.ididentifies the pull request and must be lowercase, such asgithub.com/acme/shop#42. Use the same ID in every request for the pull request. -
change_descriptionis all mabl knows about the change, because mabl doesn't read your source code. Send the pull request's title and description and the list of files it changes. -
event.occurred_atis when the event happened in your source control, in milliseconds since the Unix epoch.
To see what a stage would run without running any tests, add dry_run: true to the request.
mabl answers with what it did with the request:
{
"session_id": "[session-id]",
"created": true,
"event": "accepted",
"agent": "working"
}
event is accepted when mabl took the request, duplicate for a repeat of a request it already has, and superseded when the stage already has a newer one. Each stage keeps only its latest request, in the order of occurred_at. A late or repeated delivery can't roll a stage back to an older commit, so any request is safe to retry.
The same call covers three other kinds of requests:
- A merge queue entry. Send a request for the stage your policy defines for the merge queue, with the queue's commit as the head. Read the policy file at the merge queue's base commit.
-
A deployed commit. For a stage that runs after a merge deploys, send the request for the commit instead of a pull request. The stage needs
scope: revisionin your policy file. Send requests only for merged commits. -
A merged pull request. If your policy merges mabl branches, send a
pull_request.mergedevent when the pull request merges, with a "Command Line Interface" key.
Each case changes a few fields. Build your own client lists them.
Ask for the status
When you need the result, such as before you merge, ask mabl for the status of the stage at the commit you sent:
curl --request POST \
--url https://api.mabl.com/changeValidation/status \
--header 'accept: application/json' \
--user "key:$MABL_API_KEY" \
--header 'content-type: application/json' \
--data '
{
"workspace_id": "[workspace-id]",
"reference": {"type": "pull_request", "id": "github.com/acme/shop#42"},
"stages": [{"stage": "pull_request", "head_sha": "[head-sha]"}],
"trusted_providers": "caller"
}
'
Use the same API key that sent the stage request, with trusted_providers set to caller. mabl then judges only the requests that key sent, so nobody else can change what your pipeline is waiting on.
The response has one entry for each stage you asked about, with the stage's checks:
{
"status": "blocked",
"stages": [{
"stage": "pull_request",
"gate": "fail",
"summary": "pull_request: smoke failed, impacted passed",
"checks": [{
"id": "pull_request/web/suite/smoke",
"kind": "suite",
"must_pass": true,
"status": "failed",
"summary": "smoke: 1 of 4 plans failed",
"links": {"deployment_event": "https://app.mabl.com/..."}
}]
}]
}
The top-level status covers every stage of the pull request, so read the gate of the stage you asked about instead:
-
ready- Every check that must pass has passed or was waived. Let the change through. -
fail- A check that must pass failed, or mabl couldn't judge it. Fail the job. -
wait- A check that must pass hasn't finished. Ask again after about 20 seconds, until your own deadline.
Each check has a status and a one-line summary you can show to your team. For every status and field, see the API reference.
Post mabl's report on the pull request
To show mabl's results on the pull request itself, call POST https://api.mabl.com/changeValidation/report with the same body as the status call, plus "format": "markdown". The response includes a ready-made comment in its markdown field, with the impacted tests, the coverage gaps, and the progress of each check.
Keep one comment and update it in place. The comment's first line is <!-- mabl-change-validation -->, so you can find your earlier comment by that line. See the API reference.
When a request fails
Errors use mabl's usual error body, such as {"code": 409, "message": "...", "error_code": "change_agent_closed"}. Decide what to do from error_code, and print message in your job's log. Retry only a 429, a 5xx, or agent_stopping. Any other error repeats until you fix its cause.
| Error | What it means and what to do |
|---|---|
400 invalid_request, or no error_code
|
The request broke a rule, and message says which. Fix the request. |
403 feature_disabled
|
The account doesn't have generative AI features, or the workspace has them turned off. Skip validation. |
403 principal_too_narrow
|
The API key can't do everything validation does. A "Viewer" key gets this error. Use a "Command Line Interface" or "Editor" key. |
403 forbidden, or no error_code
|
The API key isn't allowed to send requests for the workspace. "CI/CD Integration" and "Deployment Trigger" keys get this error. Use a "Command Line Interface" or "Editor" key for the workspace. |
403 principal_gone
|
The API key that first sent a request for this pull request was deleted, disabled, or expired. Reset validation for the pull request. |
404 no_change_agent
|
mabl never accepted a stage request for this pull request in this workspace. For a merged event, there's nothing to do. For a status call, send the stage request first, and check the reference and workspace. |
409 change_agent_closed
|
Someone turned off validation for the pull request. Nothing runs until it's reset. |
409 too_many_stages
|
The pull request already has 10 stages. Use fewer stage names. |
409 another_repository
|
A request for a deployed commit names a different repository than the commit's first request. Check the repository you send. |
503 agent_stopping
|
A request to turn off validation for the pull request is still finishing. Retry the same request after a few seconds. |
| 429, other 5xx | A temporary problem. Retry the same request with backoff. |
To turn off validation for one pull request, call POST https://api.mabl.com/changeValidation/close. To start it again, or to move it to a new API key, call POST https://api.mabl.com/changeValidation/reset. Both take the workspace_id and reference of the pull request.