Skip to main content

Webhooks

Eva Ferreira
  • Updated

In mabl, you can configure your workspace to post execution details to a webhook before tests start or after tests have completed. This information can be useful if you want to chain an action in your CI/CD workflow to a specific status or outcome in your mabl tests, including:

  • Setting up resources in your cloud before mabl tests run
  • Cleaning up resources in your cloud after mabl tests finish executing

This article provides a detailed overview of mabl webhooks:

Webhook settings

You can configure webhooks from the mabl app: Settings > Webhooks. To add a new webhook, click on the + New webhook button and provide the following details:

  • Name - if you are sending mabl results to multiple webhooks, use this field to identify the purpose.
  • Base URL - mabl will call this URL with a JSON payload describing the pending or completed execution.
  • Strict SSL Validation (optional) - mabl ensures all connections use valid SSL certificates. If you’re using a CI server with a self-signed certificate, you may disable strict SSL validation.

You may configure the webhook to be sent before and/or after mabl tests run:

  • Call pre-execution - mabl sends information on the triggered tests to the webhook endpoint before test runs begin. Optionally, you can require a success response from the webhook URL. If the endpoint does not result in a successful status, the tests will be failed.
  • Call post-execution - mabl sends information on the test runs after all test executions are complete. For post-execution webhooks, indicate when mabl posts to the webhook: “on failure” (failing runs) and/or “on success” (passing runs).

Stopped runs

Tests with a stopped status do not trigger the post-execution webhook.

Troubleshooting

If you encounter issues with webhook failures, here are some common issues and suggestions for troubleshooting them.

  • Firewalls - webhook requests originate from mabl’s static IP addresses. If your webhook URL is behind a firewall, you can give mabl access by adding the mabl static IP addresses to your firewall allowlist.
  • TIMEOUT errors - each webhook must complete in 60 seconds or less. Otherwise it will fail with a TIMEOUT error.
  • 401 or 403 errors - if your webhook URL returns 401 or 403 errors, make sure that your webhooks are public so that mabl can reach them.
  • Basic auth -if you need to send basic auth credentials to your webhook, one option is to encode the credentials into the webhook URL itself. For example: https://username:password@example.com.

Webhook payloads

When a webhook event is triggered, the webhook endpoint receives a JSON payload containing information on the status, outcome and details for the specific plan(s) and test(s) that ran. All webhooks for a given workspace are fired in parallel.

The webhook payload cannot be edited.

The following attributes are included in the webhook payload:

Attribute Description
“success” The “success” attribute has a value of “true” until a state of “terminated”, “cancelled”, or “failed” is encountered.
“status” Possible status states include “queued”, “scheduling”, “scheduled”, “succeeded”, “failed”, “cancelled”, “completed.”
“status_cause” The default value for “status_cause” is null. If a test terminates in an unexpected way, such as termination by a user, or a test timeout, or another kind of error.
“plan” Information on the plan, including the plan name and labels. Ad hoc test runs will not have a plan payload.
“plan_execution”

Information on the status of that particular plan run, including status, ID, and, if available, revision.

Ad hoc test runs include information on “plan_execution” (a.k.a. plan run) despite the fact that they are not backed by an actual plan. This information is “synthetic” and is included to maintain consistency in the mabl backend.
“journeys” Information on the tests, including test name and labels.
“journey_executions” Information on the status of specific test runs, including start and stop times and any associated test case IDs.
“href” The API endpoint of the specific plan, plan run, test, or test run.
“results_href” The results URL for a specific test run in the mabl app.
“tags” The “tag” attribute is used to mark mabl auto-generated tests, such as smoke tests and link crawler tests. Tags are not user-editable.
“test_cases”

If you’ve associated a mabl test with an outside test case management tool, such as Zephyr or TM4J, the payload will include the test case ID(s) for those tests.

If you use Xray for test case management, see our article on Xray results reporting for more details on how to sync mabl test results with your Xray test cases.
“message_version” Denotes the version of the mabl webhook payload.

Journeys

Tests are referred to as “journeys” in the webhook payload.

Sample pre-execution payload

{
  "status": "queued",
  "status_cause": null,
  "success": true,
  "plan": {
    "id": "4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "name": "Print click and assert Plan",
    "labels": [ "apple", "zebra"],
    "href": "https://api.mabl.com/v1/schedule/runPolicy/4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "tags": []
  },
  "plan_execution": {
    "status": "queued",
    "status_cause": null,
    "id": "d5b96d02-c45d-4d08-8684-de55411bcede",
    "href": "https://api.mabl.com/v1/execution/runPolicyExecution/d5b96d02-c45d-4d08-8684-de55411bcede",
    "revision": "12240"
  },
  "journeys": [
    {
      "id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "name": "Print, then assert",
      "labels": [ "apple", "zebra"],
      "href": "https://api.mabl.com/v1/test/journey/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "tags": [
        "training"
      ]
    }
  ],
  "journey_executions": [],
  "start_time": null,
  "stop_time": null,
  "tags": [],
  "results": [],
  "message_version": "1.0.0"
}

Sample post-execution payload

{
  "status": "succeeded",
  "status_cause": null,
  "success": true,
  "plan": {
    "id": "4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "name": "Print click and assert Plan",
    "labels": [ "apple", "zebra"],
    "href": "https://api.mabl.com/v1/schedule/runPolicy/4ba4550d-df2b-4df2-8799-46c44dbb4b9e",
    "tags": []
  },
  "plan_execution": {
    "status": "succeeded",
    "status_cause": null,
    "id": "d5b96d02-c45d-4d08-8684-de55411bcede",
    "href": "https://api.mabl.com/v1/execution/runPolicyExecution/d5b96d02-c45d-4d08-8684-de55411bcede",
    "revision": "12240"
  },
  "journeys": [
    {
      "id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "name": "Print, then assert",
      "labels": [ "apple", "zebra"],
      "href": "https://api.mabl.com/v1/test/journey/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "tags": [
        "training"
      ]
    }
  ],
  "journey_executions": [
    {
      "status": "completed",
      "status_cause": null,
      "success": true,
      "name": "Print, then assert",
      "journey_id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "journey_execution_id": "c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "start_time": 1517612117504,
      "stop_time": 1517612146280,
      "href": "https://api.mabl.com/v1/execution/runPolicyExecution/4ba4550d-df2b-4df2-8799-46c44dbb4b9e/testScriptExecution/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "results_href": "https://app.mabl.com/workspaces/3627972b-a762-4a87-a3fb-97cde3e9c414/test/plan-executions/d5b96d02-c45d-4d08-8684-de55411bcede/journeys/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0",
      "test_cases": [
        {
          "id": "MABL-1234"
        },
        {
          "id": "MABL-999"
        }
      ]
    }
  ],
  "start_time": 1517612110113,
  "stop_time": 1517612146368,
  "tags": [],
  "results": [
    {
      "success": true,
      "name": "Print, then assert",
      "results_href": "https://app.mabl.com/workspaces/3627972b-a762-4a87-a3fb-97cde3e9c414/test/plan-executions/d5b96d02-c45d-4d08-8684-de55411bcede/journeys/c221c1f8-aa52-4fdb-a1bd-5ee5b408ac9f:0"
    }
  ],
  "message_version": "1.0.0"
}

Related articles