API-based CI/CD integration

Suggest Edits

Most mabl customers use mabl's existing integrations or the mabl CLI to integrate end-to-end testing into their DevOps pipeline. However, If none of those options fit your needs, you can use the mabl APIs to run end-to-end tests on code deployment.

📘
Recommended reading
Before integrating mabl into your CI/CD pipeline, please review the following resources:

Test execution overview

Mabl deployment events

Continuous testing in CI/CD

This guide provides you with an example of how to integrate mabl as a step in your CI/CD pipeline using the Deployment Events API:

The first API call triggers all plans associated with a particular environment, application and/or plan labels in response to a new software deployment event.
The second API call checks the status of the plans that were launched as a result of the first API call.

The provided examples assume that you are using a CI build tool like Jenkins, Gitlab, or CircleCI and a Linux-based container which allows you to write custom build steps in bash with standard tools such as curl and jq. If you are using a different type of build tool you may need to adapt these examples.

Hypothetical CI/CD workflow

Imagine that your team has an automated CI/CD workflow into which you want to integrate end-to-end testing with mabl. For example, consider the following pipeline, which represents part of a CI/CD workflow:

Create deployment event

The first step in adding mabl to your CI/CD flow is to notify mabl when you deploy your application using a mabl deployment event. When you invoke the create deployment event endpoint, mabl searches for all plans associated with the specified environment, application and/or labels that contain a deployment trigger and then executes those plans immediately.

Before you can get started with the Deployment Events API you'll need a few pieces of information:

A deployment trigger API key. (See Managing API keys for more info.)
At least one of the following mabl resource IDs:
- Environment ID
- Application ID
Plan labels: if you want to target specific plans

Building a request

You can build a curl command to trigger a deployment event on the API page in the mabl app:

Navigate to Settings > APIs.
Scroll down to API documentation.
Select "Create deployment event" from the dropdown.
Add the request details to generate a curl command.

Optional information

In addition to the minimum required environment_id or application_id properties, you can also pass other reference information associated with the deployment event. Here is a more complete example of a JSON message that can be passed to the API:

{
  "environment_id":"FQWY5iMcfkmqV64YMyrt5w-e",
  "revision":"937e77b3822a4ae2a38135e08a5d4a8eb6eddfdd",
  "event_time":1522778187000,
  "properties":{
    repository_branch_name: 'master',
    repository_commit_username: 'mablcoder',
    repository_url: '[email protected]:mycompany/myrepo.git',
    repository_pull_request_url: 'https://github.com/mycompany/myrepo/pull/1013',
    repository_pull_request_number: 1013,
    repository_pull_request_title: 'good pr'
  }
}

The revision, event_time, and properties attributes are all optional. Here is a quick description of what each of these mean:

revision: The SCM revision associated with the deployment, e.g. a git hash
event_time: The time when the deployment event occurred, as a UNIX epoch timestamp in milliseconds. It may be desirable to pass this if there is a delay between when the deployment occurs and the Deployment Events API is invoked
properties: Arbitrary map of key-value pairs. You can put any additional information here. There are a few standard properties that mabl recognizes and associates special meaning with. Many of our native integrations provide these properties by default:
- repository_branch_name: The code branch this deployment was build from.
- repository_commit_username: The username of the user who committed the code being deployed
- repository_url: The url of the code repository that triggered this deployment
- repository_pull_request_url: The url of the pull request that triggered this deployment.
- repository_pull_request_number: The number of the pull request that triggered this deployment.
- repository_pull_request_title: The title of the pull request that triggered this deployment.

Triggering a deployment event

When you trigger a deployment event, you get a JSON message back containing an ID and some other information. For example, it might look like this:

{
  "id": "OCn2EccmXk9eFvRInTfwkQ-v",
  "environment_id": "dbd62e2e-bbbc-4c15-ab7c-72aa0ce379fe",
  "received_time": 1522777753410
}

The most important part of the response is the id field. This value is the deployment event ID, and. you can use it to get the results of the tests that were triggered by the deployment event.

Get results of the deployment event

After creating a deployment event, you can check the status of plans that are launched by using the get a deployment result summary endpoint.

Building a request

You can build a curl command to retrieve the status of a deployment event on the API page in the mabl app:

Navigate to Settings > APIs.
Scroll down to API documentation.
Select "Get results from a deployment event" from the dropdown.
Enter the deployment event ID.

Getting a response

The get a deployment result summary endpoint returns a JSON object containing details about the status of all plans that were triggered by the deployment event.

Here is a sample response:

{
  "event_status": {
    "succeeded": true,
    "succeeded_first_attempt": true
  },
  "plan_execution_metrics": {
    "total": 1,
    "passed": 1,
    "failed": 0
  },
  "journey_execution_metrics": {
    "total": 1,
    "passed": 1,
    "failed": 0
  },
  "executions": [
    {
      "status": "succeeded",
      "success": true,
      "plan": {
        "id": "c953e3e3-a195-4b36-bdf8-49ec73ee3d35",
        "name": "Verify home page load and login",
        "label": "regression",
        "href": "https://api.mabl.com/schedule/runPolicy/c953e3e3-a195-4b36-bdf8-49ec73ee3d35",
        "app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/test/plans/c953e3e3-a195-4b36-bdf8-49ec73ee3d35"
      },
      "plan_execution": {
        "id": "I9wjeolJE-iuvOmCSF28Lw-pe",
        "status": "succeeded",
        "href": "https://api.mabl.com/execution/runPolicyExecution/c953e3e3-a195-4b36-bdf8-49ec73ee3d35"
      },
      "journeys": [
        {
          "id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "name": "Visit home page",
          "href": "https://api.mabl.com/execution/runPolicyExecution/I9wjeolJE-iuvOmCSF28Lw-pe/testScriptExecution/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/test/plan-executions/I9wjeolJE-iuvOmCSF28Lw-pe/journeys/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0"
        }
      ],
      "journey_executions": [
        {
          "journey_id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "journey_execution_id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "status": "completed",
          "success": true,
          "href": "https://api.mabl.com/test/journey/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/train/journeys/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "test_cases": [
            {
              "id": "MABL-1234"
            },
            {
              "id": "MABL-999"
            }
          ]
        }
      ],
      "start_time": 1522777753987,
      "stop_time": 1522777997841
    }
  ]
}

For most purposes it should be sufficient to process only the summary information under plan_execution_metrics_and journey_execution_metrics. Any output field named app_href contains a link to the mabl UI that may be used to directly view an entity or the results of an execution.

📘
Tests used to be called journeys
The API response still reflects the old term for tests: "journey".

Inserting mabl into your pipeline

One option for inserting mabl into your CI/CD pipeline would be to:

Trigger a deployment event to run certain plans when a deployment occurs to a particular environment.
Poll the endpoint for retrieving the results of a deployment event until all plans have either passed or failed.

Using the hypothetical CI/CD workflow from earlier, it might look something like this:

Tying it all together

The following bash script can trigger a deployment event and then poll the executions until they either complete or fail. This sample script takes the API key and environment ID as arguments: sh mabl-deployment-integration.sh api-key env-id.

#!/bin/bash

MABL_API_BASE="https://api.mabl.com"
POLL_SEC=10

if [ "$#" -ne 2 ]; then
  echo "Usage: $0 <api-key> <environment-id>"
  exit 0
fi

API_KEY="$1"
ENV_ID="$2"

# Create a deployment event, and retrieve its ID:
deployment_event=$(curl -s "${MABL_API_BASE}/events/deployment" -u "key:${API_KEY}" -H 'Content-Type:application/json' -d "{\"environment_id\":\"${ENV_ID}\"}")
event_id=$(echo "${deployment_event}" | jq -r '.id')
echo "Deployment Event ID: ${event_id}"

# Poll the Execution Result API until all Plans have completed
while true; do
  echo "Waiting for executions to complete..."
  sleep ${POLL_SEC}
  results=$(curl -s "${MABL_API_BASE}/execution/result/event/${event_id}" -u "key:${API_KEY}")
  event_status=$(echo "${results}" | jq '.event_status')
  if [ -z "${event_status}" ] || [ "${event_status}" = "{}" ]; then
    continue
  fi

  plan_metrics=$(echo "${results}" | jq '.plan_execution_metrics')
  if [ "${plan_metrics}" != "null" ]; then
    total_plans=$(echo "${plan_metrics}" | jq -r '.total')
    passed_plans=$(echo "${plan_metrics}" | jq -r '.passed')
    failed_plans=$(echo "${plan_metrics}" | jq -r '.failed')
  fi

  succeded=$(echo "${event_status}" | jq -r '.succeeded')
  succeded_with_retries=$(echo "${event_status}" | jq -r '.succeeded_with_retries')
  break
done

# Print summary:
echo
echo "Full Results:"
echo "${results}" | jq
echo
echo "Total plans executed: ${total_plans} succeded: ${passed_plans} failed: ${failed_plans}"
if [ "${succeded}" = "true" ]; then
  if [ "${succeded_with_retries}" = "true" ]; then
    echo "Successful run with retries"
  else
    echo "Successful run"
  fi
  exit 0
else
  echo "One or more plans have failed"
  exit 1
fi

Updated 5 months ago

Related resources