Skip to main content

Webhooks

Eva Ferreira
  • Updated

In mabl, you can configure your workspace to post execution details to a webhook before tests start or after tests have completed. This information can be useful if you want to chain an action in your CI/CD workflow to a specific status or outcome in your mabl tests, including:

  • Setting up resources in your cloud before mabl tests run
  • Cleaning up resources in your cloud after mabl tests finish executing

This article provides a detailed overview of mabl webhooks:

Webhook settings

You can configure webhooks from the mabl app: Settings > Webhooks. Creating and editing webhooks requires the editor role or above. To add a new webhook, click on the + New webhook button and provide the following details:

  • Name - if you are sending mabl results to multiple webhooks, use this field to identify the purpose.
  • Base URL - mabl will call this URL with a JSON payload describing the pending or completed execution.
  • Strict SSL Validation (optional) - mabl ensures all connections use valid SSL certificates. If you’re using a CI server with a self-signed certificate, you may disable strict SSL validation.
  • Custom HTTP headers (optional) - if your receiver sits behind an API gateway or a proxy that checks a token, click on Add header and enter the name and value of each header it expects. mabl sends them with every request to this webhook. See custom header rules for more details.

You may configure the webhook to be sent before and/or after mabl tests run:

  • Call pre-execution - mabl sends information on the triggered tests to the webhook endpoint before test runs begin. Optionally, you can require a success response from the webhook URL. If the endpoint does not result in a successful status, the tests will be failed.
  • Call post-execution - mabl sends information on the test runs after all test executions are complete. For post-execution webhooks, indicate when mabl posts to the webhook: “on failure” (failing runs) and/or “on success” (passing runs).

Stopped runs

Tests with a stopped status do not trigger the post-execution webhook.

Custom header rules

  • A webhook can carry up to 20 headers, and each header name can appear only once.
  • Header values are stored encrypted. mabl shows them only when you open the webhook to edit it.
  • Header names and values must use plain ASCII characters.
  • mabl rejects the headers it sets on the request itself, such as Content-Type and User-Agent, and hop-by-hop headers such as Transfer-Encoding.

Edit a webhook

To change a webhook, go to Settings > Webhooks and click on the webhook. On the Edit webhook page, you can change the URL, the custom HTTP headers, and the pre- and post-execution settings, then save. Changes apply to the next request mabl sends.

Troubleshooting

If you encounter issues with webhook failures, here are some common issues and suggestions for troubleshooting them.

  • Firewalls - webhook requests originate from mabl’s static IP addresses. If your webhook URL is behind a firewall, you can give mabl access by adding the mabl static IP addresses to your firewall allowlist.
  • TIMEOUT errors - each webhook must complete in 60 seconds or less. Otherwise it will fail with a TIMEOUT error.
  • 401 or 403 errors - your receiver rejected the request. If it expects a token or an API key, add the header it checks, such as Authorization, as a custom HTTP header. If it uses basic auth, you can encode the credentials into the webhook URL itself, for example https://username:password@example.com.

Webhook payloads

When a webhook event is triggered, the webhook endpoint receives a JSON payload containing information on the status, outcome and details for the specific plan(s) and test(s) that ran. All webhooks for a given workspace are fired in parallel.

The webhook payload cannot be edited.

The following attributes are included in the webhook payload:

Attribute Description
“success” The “success” attribute has a value of “true” until a state of “terminated”, “cancelled”, or “failed” is encountered.
“status” Possible status states include “queued”, “scheduling”, “scheduled”, “succeeded”, “failed”, “cancelled”, “completed.”
“status_cause” The default value for “status_cause” is null. If a test terminates in an unexpected way, such as termination by a user, or a test timeout, or another kind of error.
“plan” Information on the plan, including the plan name and labels. Ad hoc test runs will not have a plan payload.
“plan_execution” Information on the status of that particular plan run, including status, ID, and, if available, revision.
Ad hoc test runs include information on “plan_execution” (a.k.a. plan run) despite the fact that they are not backed by an actual plan. This information is “synthetic” and is included to maintain consistency in the mabl backend.
“journeys” Information on the tests, including test name and labels.
“journey_executions” Information on the status of specific test runs, including start and stop times and any associated test case IDs.
“href” The API endpoint of the specific plan, plan run, test, or test run.
“results_href” The results URL for a specific test run in the mabl app.
“tags” The “tag” attribute is used to mark mabl auto-generated tests, such as smoke tests and link crawler tests. Tags are not user-editable.
“test_cases” If you’ve associated a mabl test with an outside test case management tool, such as Zephyr or TM4J, the payload will include the test case ID(s) for those tests.
If you use Xray for test case management, see our article on Xray results reporting for more details on how to sync mabl test results with your Xray test cases.
“message_version” Denotes the version of the mabl webhook payload.

Journeys

Tests are referred to as “journeys” in the webhook payload.

Sample pre-execution payload

{
  "status": "queued",
  "status_cause": null,
  "success": true,
  "plan": {
    "id": "4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "name": "Print click and assert Plan",
    "labels": [ "apple", "zebra"],
    "href": "https://api.mabl.com/v1/schedule/runPolicy/4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "tags": []
  },
  "plan_execution": {
    "status": "queued",
    "status_cause": null,
    "id": "d5b96d02-c45d-4d08-8684-de55411bcede",
    "href": "https://api.mabl.com/v1/execution/runPolicyExecution/d5b96d02-c45d-4d08-8684-de55411bcede",
    "revision": "12240"
  },
  "journeys": [
    {
      "id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "name": "Print, then assert",
      "labels": [ "apple", "zebra"],
      "href": "https://api.mabl.com/v1/test/journey/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "tags": [
        "training"
      ]
    }
  ],
  "journey_executions": [],
  "start_time": null,
  "stop_time": null,
  "tags": [],
  "results": [],
  "message_version": "1.0.0"
}

Sample post-execution payload

{
  "status": "succeeded",
  "status_cause": null,
  "success": true,
  "plan": {
    "id": "4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "name": "Print click and assert Plan",
    "labels": [ "apple", "zebra"],
    "href": "https://api.mabl.com/v1/schedule/runPolicy/4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "tags": []
  },
  "plan_execution": {
    "status": "succeeded",
    "status_cause": null,
    "id": "d5b96d02-c45d-4d08-8684-de55411bcede",
    "href": "https://api.mabl.com/v1/execution/runPolicyExecution/d5b96d02-c45d-4d08-8684-de55411bcede",
    "revision": "12240"
  },
  "journeys": [
    {
      "id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "name": "Print, then assert",
      "labels": [ "apple", "zebra"],
      "href": "https://api.mabl.com/v1/test/journey/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "tags": [
        "training"
      ]
    }
  ],
  "journey_executions": [
    {
      "status": "completed",
      "status_cause": null,
      "success": true,
      "name": "Print, then assert",
      "journey_id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "journey_execution_id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "start_time": 1517612117504,
      "stop_time": 1517612146280,
      "href": "https://api.mabl.com/v1/execution/runPolicyExecution/4ba4550d-df2b-4df2-8799-46c44dbb4b9e/testScriptExecution/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "results_href": "https://app.mabl.com/workspaces/3627972b-a762-4a87-a3fb-97cde3e9c414/test/plan-executions/d5b96d02-c45d-4d08-8684-de55411bcede/journeys/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "test_cases": [
        {
          "id": "MABL-1234"
        },
        {
          "id": "MABL-999"
        }
      ]
    }
  ],
  "start_time": 1517612110113,
  "stop_time": 1517612146368,
  "tags": [],
  "results": [
    {
      "success": true,
      "name": "Print, then assert",
      "results_href": "https://app.mabl.com/workspaces/3627972b-a762-4a87-a3fb-97cde3e9c414/test/plan-executions/d5b96d02-c45d-4d08-8684-de55411bcede/journeys/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0"
    }
  ],
  "message_version": "1.0.0"
}

Related articles