API-based CI/CD integration
Most mabl customers use mabl's existing integrations or the mabl CLI to integrate end-to-end testing into their DevOps pipeline. However, If none of those options fit your needs, you can use the mabl APIs to run end-to-end tests on code deployment.
Recommended reading
Before integrating mabl into your CI/CD pipeline, please review the different ways to run mabl tests and check out the article on Continuous testing in CI/CD
This article provides you with an example of how to integrate mabl as a step in your CI/CD pipeline using two simple API calls. The first API call will kick off all plans associated with a particular environment, application and/or labels in response to a new software deployment event. The second API call can be used to check the status of the plans that were launched as a result of the first API call.
The provided examples assume that you are using a CI build tool like Jenkins, Gitlab, or CircleCI and a Linux-based container which allows you to write custom build steps in bash with standard tools such as curl and jq. If you are using a different type of build tool you may need to adapt these examples.
Hypothetical CI/CD workflow
Let's assume that your team has an automated CI/CD workflow into which you want to integrate end-to-end testing with mabl. As an example, consider the following simple pipeline which represents part of a CI/CD workflow:
Deployment Events API
The first step in adding mabl to your CI/CD flow is to notify mabl when you deploy your application using the Deployment Events API. When you invoke the Deployment Events API, mabl will search for all plans associated with the specified environment, application and/or labels that contain a deployment trigger, and then it will execute those plans immediately.
Before you can get started with the Deployment Events API you'll need a few pieces of information:
- A deployment trigger API key. (See Managing API keys for more info.)
- At least one of the following IDs:
- mabl Environment ID
- mabl Application ID
- Plan labels (if you want to target specific plans)
Plan labels
Labels are a great way to organize your mabl Plans and you can pass them as parameters to the API for more granular control over which plans get executed.
Curl command builder
One way to learn how to use the Deployment Event API is by navigating to Settings => APIs in mabl and selecting Deployment Events API under the API documentation section. Use the CURL command builder to specify the plans you'd like to trigger. You can find more information on this feature here.
Interactive API Reference
You can also play with the APIs using the interactive API reference.
In addition to the minimal environment_id property, you can also pass other reference information associated with the deployment event. Here is a more complete example of a JSON message that can be passed to the API:
{
"environment_id":"FQWY5iMcfkmqV64YMyrt5w-e",
"revision":"937e77b3822a4ae2a38135e08a5d4a8eb6eddfdd",
"event_time":1522778187000,
"properties":{
repository_branch_name: 'master',
repository_commit_username: 'mablcoder',
repository_url: '[email protected]:mycompany/myrepo.git',
repository_pull_request_url: 'https://github.com/mycompany/myrepo/pull/1013',
repository_pull_request_number: 1013,
repository_pull_request_title: 'good pr'
}
}
The revision, event_time, and properties attributes are all optional. Here is a quick description of what each of these mean:
- revision: The SCM revision associated with the deployment, e.g. a git hash
- event_time: The time when the deployment event occurred, as a UNIX epoch timestamp in milliseconds. It may be desirable to pass this if there is a delay between when the deployment occurs and the Deployment Events API is invoked
- properties: Arbitrary map of key-value pairs. You can put any additional information here. There are a few standard properties that mabl recognizes and associates special meaning with. Many of our native integrations provide these properties by default. They are...
- repository_branch_name: The code branch this deployment was build from.
- repository_commit_username: The username of the user who committed the code being deployed
- repository_url: The url of the code repository that triggered this deployment
- repository_pull_request_url: The url of the pull request that triggered this deployment.
- repository_pull_request_number: The number of the pull request that triggered this deployment.
- repository_pull_request_title: The title of the pull request that triggered this deployment.
Triggering a deployment event
When you invoke the Deployment Events API you will get a JSON message back containing an event ID and some other information. For example, it might look like this:
{
"id": "OCn2EccmXk9eFvRInTfwkQ-v",
"environment_id": "dbd62e2e-bbbc-4c15-ab7c-72aa0ce379fe",
"received_time": 1522777753410
}
The most important part of the response is the id field. This value is the deployment event ID and will be used later when we want to retrieve the results of the executions that were kicked off my our deployment event.
Execution Result API
After a deployment event has been created, the status of plans that were launched as a result of that event can be interrogated using the Execution Result API. In order to use the Execution Result API you must first have a deployment event ID (see the previous section on "Triggering a deployment event").
You can build a CURL command to retrieve the status of a deployment event on the API page in the mabl app:
- Navigate to
Settings > APIs. - Scroll down to API documentation.
- Select "Execution Result API" from the dropdown. A CURL command builder for the Execution Result API will appear.
- Enter the deployment event ID.
- Copy and paste the CURL command into your terminal.
- Press enter. The Execution Result API will return a JSON object containing summary information about all plans and tests that were executed including detailed information about each one.
Here is a sample response from the Execution Result API:
{
"event_status": {
"succeeded": true,
"succeeded_first_attempt": true
},
"plan_execution_metrics": {
"total": 1,
"passed": 1,
"failed": 0
},
"journey_execution_metrics": {
"total": 1,
"passed": 1,
"failed": 0
},
"executions": [
{
"status": "succeeded",
"success": true,
"plan": {
"id": "c953e3e3-a195-4b36-bdf8-49ec73ee3d35",
"name": "Verify home page load and login",
"label": "regression",
"href": "https://api.mabl.com/schedule/runPolicy/c953e3e3-a195-4b36-bdf8-49ec73ee3d35",
"app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/test/plans/c953e3e3-a195-4b36-bdf8-49ec73ee3d35"
},
"plan_execution": {
"id": "I9wjeolJE-iuvOmCSF28Lw-pe",
"status": "succeeded",
"href": "https://api.mabl.com/execution/runPolicyExecution/c953e3e3-a195-4b36-bdf8-49ec73ee3d35"
},
"journeys": [
{
"id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
"name": "Visit home page",
"href": "https://api.mabl.com/execution/runPolicyExecution/I9wjeolJE-iuvOmCSF28Lw-pe/testScriptExecution/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
"app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/test/plan-executions/I9wjeolJE-iuvOmCSF28Lw-pe/journeys/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0"
}
],
"journey_executions": [
{
"journey_id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
"journey_execution_id": "6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
"status": "completed",
"success": true,
"href": "https://api.mabl.com/test/journey/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
"app_href": "https://app.mabl.com/workspaces/5bf0ebb1-f158-4c02-9e71-1be9f6ce7d17/train/journeys/6b194f04-1ea0-45f1-8a1a-04c9338067d9:0",
"test_cases": [
{
"id": "MABL-1234"
},
{
"id": "MABL-999"
}
]
}
],
"start_time": 1522777753987,
"stop_time": 1522777997841
}
]
}
For most purposes it should be sufficient to process only the summary information under plan_execution_metrics and journey_execution_metrics. Any output field named app_href contains a link to the mabl UI that may be used to directly view an entity or the results of an execution.
Tests used to be called Journeys
Our API response still reflects the old term "Journey."
Inserting mabl into your pipeline
One option for inserting mabl into your CI/CD pipeline would be to:
- Kick off certain Plans when a deployment occurs to a particular environment using the Deployment Events API
- Poll the Execution Result API until all plans have either passed or failed
Using the earlier pipeline example, it might look something like this:
Tying it all together
Below is a bash script that can trigger a deployment event and then poll the executions until they either complete or fail. This sample script takes the API key and environment ID as arguments: sh mabl-deployment-integration.sh api-key env-id.
#!/bin/bash
MABL_API_BASE="https://api.mabl.com"
POLL_SEC=10
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <api-key> <environment-id>"
exit 0
fi
API_KEY="$1"
ENV_ID="$2"
# Create a deployment event, and retrieve its ID:
deployment_event=$(curl -s "${MABL_API_BASE}/events/deployment" -u "key:${API_KEY}" -H 'Content-Type:application/json' -d "{\"environment_id\":\"${ENV_ID}\"}")
event_id=$(echo "${deployment_event}" | jq -r '.id')
echo "Deployment Event ID: ${event_id}"
# Poll the Execution Result API until all Plans have completed
while true; do
echo "Waiting for executions to complete..."
sleep ${POLL_SEC}
results=$(curl -s "${MABL_API_BASE}/execution/result/event/${event_id}" -u "key:${API_KEY}")
event_status=$(echo "${results}" | jq '.event_status')
if [ -z "${event_status}" ] || [ "${event_status}" = "{}" ]; then
continue
fi
plan_metrics=$(echo "${results}" | jq '.plan_execution_metrics')
if [ "${plan_metrics}" != "null" ]; then
total_plans=$(echo "${plan_metrics}" | jq -r '.total')
passed_plans=$(echo "${plan_metrics}" | jq -r '.passed')
failed_plans=$(echo "${plan_metrics}" | jq -r '.failed')
fi
succeded=$(echo "${event_status}" | jq -r '.succeeded')
succeded_with_retries=$(echo "${event_status}" | jq -r '.succeeded_with_retries')
break
done
# Print summary:
echo
echo "Full Results:"
echo "${results}" | jq
echo
echo "Total plans executed: ${total_plans} succeded: ${passed_plans} failed: ${failed_plans}"
if [ "${succeded}" = "true" ]; then
if [ "${succeded_with_retries}" = "true" ]; then
echo "Successful run with retries"
else
echo "Successful run"
fi
exit 0
else
echo "One or more plans have failed"
exit 1
fi
Updated about 1 month ago