Integrating mabl with your CI/CD workflow

Integrating mabl in your DevOps workflow enables you to rigger automated tests on deployment which helps to quickly validate that new code doesn't cause any regressions as it makes its way across environments such as staging and production. It also gives peace of mind to QA engineers, product managers, and other business stakeholders that any issues are proactively resolved and everything continues to operate as expected. Some bugs may only reveal themselves in a production environment, so it’s great when we can run tests every deployment.

Integrating mabl into your CI/CD pipeline

You can integrate mabl as a step in your CI/CD pipeline using two simple API calls. The first API call will kick off all Plans associated with a particular Environment, Application and/or Labels in response to a new software deployment event. The second API call can be used to check the status of the Plans that were launched as a result of the first API call.

Examples used in this documentation

These examples will assume that you are using a build tool like Codeship or CircleCI and a Linux-based container which allows you to write custom build steps in bash with standard tools such as curl and jq. If you are using a different type of build tool you may need to adapt these examples.

Hypothetical CI/CD workflow

Here we will assume that your team has an automated CI/CD workflow into which you could integrate mabl. As an example, consider the following simple pipeline which represents part of a CI/CD workflow:

Deployment Events API

The first step in adding mabl to your CI/CD flow is to notify mabl when you deploy your application using the Deployment Events API. When you invoke the Deployment Events API mabl will search for all plans associated with the specified Environment, Application and/or Labels that contain a Deployment trigger, and then it will execute those Plans immediately.

Add a "deployment" trigger to any plans that you want to run after a deployment

Before you can get started with the Deployment Event API you'll need a few pieces of information:

  • Your workspace's API key, and
  • One of the following IDs:
    • mabl Environment ID
    • mabl Application ID
    • mabl Environment ID and/or Application ID and/or Plan Labels

Labels are a great way to organize your mabl Plans and you can pass them as parameters to the API for more granular control over which Plans get executed. However, we recommend starting with just an Environment ID for simplicity.

The easiest way to learn how to use the Deployment Event API is by navigating to Settings => APIs in mabl and selecting Deployment Events API under the API documentation section. You can then select a mabl Environment and see the resulting curl command for invoking the Deployment Events API:

In addition to the minimal environment_id property, you can also pass other reference information associated with the deployment event. Here is a more complete example of a JSON message that can be passed to the API:

{
  "environment_id":"FQWY5iMcfkmqV64YMyrt5w-e",
  "revision":"937e77b3822a4ae2a38135e08a5d4a8eb6eddfdd",
  "event_time":1522778187000,
  "properties":{
    "committer":"employee@example.com"
  }
}

The revision, event_time, and properties attributes are all optional. Here is a quick description of what each of these mean:

  • revision: The SCM revision associated with the deployment, e.g. a git hash
  • event_time: The time when the deployment event occurred, as a UNIX epoch timestamp in milliseconds. It may be desirable to pass this if there is a delay between when the deployment occurs and the Deployment Events API is invoked
  • properties: Arbitrary map of key-value pairs. You can put any additional information here.

When you invoke the Deployment Events API you will get a JSON message back containing an event ID and some other information. For example, it might look like this:

{
  "id": "OCn2EccmXk9eFvRInTfwkQ-v",
  "environment_id": "dbd62e2e-bbbc-4c15-ab7c-72aa0ce379fe",
  "received_time": 1522777753410
}

The most important part of the response is the id field. The event ID will need to be used later when we want to retrieve the results of the executions that were kicked off my our deployment event.

Execution Result API

After a Deployment Event has been created, the status of Plans that were launched as a result of that event can be interrogated using the Execution Result API. In order to use the Execution Result API you must first have a Deployment Event ID (see Deployment Events API above).

Just like the Deployment Events API, the easiest way to learn about the Execution Result API is to use the curl builder in mabl. Navigate to Settings => APIs, and select Execution Result API under the API documentation section.

The Execution Result API will return a JSON object containing summary information about all Plans and Journeys that were executed including detailed information about each one. Here is an example response:

{
  "plan_execution_metrics": {
    "total": 1,
    "passed": 1,
    "failed": 0
  },
  "journey_execution_metrics": {
    "total": 1,
    "passed": 1,
    "failed": 0
  },
  "executions": [
    {
      "status": "succeeded",
      "success": true,
      "plan": {
        "id": "c953e3e3-a195-4b36-bdf8-49ec73ee3d35",
        "name": "Verify home page load and login",
        "href": "https://api.mabl.com/schedule/runPolicy/c953e3e3-a195-4b36-bdf8-49ec73ee3d35",
        "app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/test/plans/c953e3e3-a195-4b36-bdf8-49ec73ee3d35"
      },
      "plan_execution": {
        "id": "I9wjeolJE-iuvOmCSF28Lw-pe",
        "status": "succeeded",
        "href": "https://api.mabl.com/execution/runPolicyExecution/c953e3e3-a195-4b36-bdf8-49ec73ee3d35"
      },
      "journeys": [
        {
          "id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "name": "Visit home page",
          "href": "https://api.mabl.com/execution/runPolicyExecution/I9wjeolJE-iuvOmCSF28Lw-pe/testScriptExecution/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/test/plan-executions/I9wjeolJE-iuvOmCSF28Lw-pe/journeys/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0"
        }
      ],
      "journey_executions": [
        {
          "journey_id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "journey_execution_id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "status": "completed",
          "success": true,
          "href": "https://api.mabl.com/test/journey/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
          "app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/train/journeys/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0"
        }
      ],
      "start_time": 1522777753987,
      "stop_time": 1522777997841
    }
  ]
}

For most purposes it should be sufficient to process only the summary information under plan_execution_metrics and journey_execution_metrics. Any output field named app_href contains a link to the mabl UI that may be used to directly view an entity or the results of an execution.

Inserting mabl into your pipeline

One option for inserting mabl into your CI/CD pipeline would be to:

  1. Kick off certain Plans when a deployment occurs to a particular environment using the Deployment Events API
  2. Poll the Execution Result API until all plans have either passed or failed

Using the earlier pipeline example, it might look something like this:

Tying it all together

Below is a bash script that can be used to trigger a deployment event and then poll the executions until they either complete or fail. Of course you will need to substitute your own API key and environment ID before using or adapting this example.

Be sure to replace 'api-key' and 'environment-id' below with your own API key and environment ID.

#!/bin/bash

MABL_API_BASE="https://api.mabl.com"
POLL_SEC=10

if [ "$#" -ne 2 ]; then
  echo "Usage: $0 <api-key> <environment-id>"
  exit 0
fi

API_KEY="$1"
ENV_ID="$2"

# Create a deployment event, and retrieve its ID:
deployment_event=`curl -s "${MABL_API_BASE}/events/deployment" -u "key:${API_KEY}" -H 'Content-Type:application/json' -d "{\"environment_id\":\"${ENV_ID}\"}"`
event_id=`echo ${deployment_event} | jq -r '.id'`
echo "Deployment Event ID: ${event_id}"

# Poll the Execution Result API until all Plans have completed
complete=false
while [ ${complete} == false ]; do
  echo "Waiting for executions to complete..."
  sleep ${POLL_SEC}
  results=`curl -s "${MABL_API_BASE}/execution/result/event/${event_id}" -u "key:${API_KEY}"`
  plan_metrics=`echo ${results} | jq '.plan_execution_metrics'`
  if [ "${plan_metrics}" == "null" ]; then
    continue
  fi
  total_plans=`echo ${plan_metrics} | jq -r '.total'`
  passed_plans=`echo ${plan_metrics} | jq -r '.passed'`
  failed_plans=`echo ${plan_metrics} | jq -r '.failed'`
  let "passed_and_failed_plans = ${passed_plans} + ${failed_plans}"
  if [ "${passed_and_failed_plans}" -eq "${total_plans}" ]; then
    complete=true
  fi
done

# Print summary:
echo
echo "Full Results:"
echo ${results} | jq
echo
if [ "${failed_plans}" -gt "0" ]; then
  echo "${failed_plans} plans failed!"
else
  echo "All plans passed."
fi
echo

Integrating mabl with your CI/CD workflow


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.