Zephyr integration

With the mabl integration for Zephyr, your team can keep an up-to-date record of mabl test results in Zephyr, so you have the evidence you need for release reporting and compliance audits. Capabilities include:

• Syncing mabl plan run results to Zephyr test cases
• Creating new mabl tests based on Zephyr test case requirements

This article explains how to set up the Zephyr integration, sync mabl results to Zephyr test cases, and create mabl tests from Zephyr test cases.

Set up the integration

Gather your Zephyr details

Before you add the integration in mabl, collect the following from Zephyr and Jira:

1. The URL of the Jira site that Zephyr runs on, such as https://your-site.atlassian.net.
2. The region your Zephyr site is hosted in: United States, Europe, Australia, or Germany.
3. A Zephyr API token (see generation steps below).
4. The keys of the Jira projects whose Zephyr test cases you want to link. For example, if your test cases take the form of PROJ-T123, the project key is PROJ.

To generate an API token:

1. Open your personal settings in Jira: click on the settings (cog) icon in the top navigation bar and select General settings.
2. Select Zephyr API Access Tokens from the left navigation panel.
3. Click Create access token and copy the key immediately. Note that a token only works in its own hosting region.

Add the integration in mabl

1. In the mabl app, go to the integrations page: Settings > Integrations.
2. In the Available integrations section, click on the + Setup button for Zephyr.
3. Give the integration a name. The default name is "Zephyr".
4. Under Setup, leave Integration enabled on. Check Automatically sync plan run results if you want mabl to record every completed plan run in Zephyr on its own. You can also turn it on later. See Automatic sync for details.
5. Enter the Jira site URL, choose the Zephyr region, and enter the API token.
6. In Zephyr project keys, enter the keys of the projects whose test cases you want to link, such as PROJ. Separate multiple keys with commas. mabl only records results in these projects.
7. Click Test connection. mabl checks the token, projects, and region against Zephyr and, on success, loads the statuses your projects share.
8. Under Test cycles, choose where mabl records each plan run's results:
   • Create a test cycle for each plan run: mabl creates a new test cycle for every synced plan run. Optionally, enter a folder ID for each project's test cycles. Leave it blank to create test cycles at the top level.
   • Add results to an existing test cycle: enter the key of an existing test cycle for each project. Every synced plan run adds its executions to that cycle.
9. Under Execution statuses, choose the Zephyr status to record for Passed, Failed, and Skipped mabl tests. If you don't want skipped tests recorded, choose Don't record skipped tests.
10. Under Result details, choose what each execution carries:
    • Attach a screenshot to failed executions: mabl attaches the final application state screenshot of the failing test run.
    • Record the mabl environment: when the Zephyr project has an environment with the same name as the mabl environment, mabl records it on the execution.
11. Click Save.

On saving, the integration appears in the Active integrations section with its projects and region.

Edit, disable, or remove the integration

To edit the integration, find it in the Active integrations section and click on the pencil icon. mabl never shows the saved API token again; the field reads "API token (saved)".

• To change other settings, leave the API token field empty. mabl will keep using the saved token.
• To replace the token, enter a new one and click Save.

To pause the integration without removing it, turn off Integration enabled and click Save. While the integration is disabled, mabl doesn't sync results to Zephyr.

To remove the integration, click on the trash icon in its row and confirm. Results stop syncing, and the linked case keys stay on your tests as plain text in the Test Case ID(s) field. If you add the integration back, the keys show as linked cases again and results resume syncing.

Reconnect the integration

If the integration shows a Reconnect required status in the Active integrations section, Zephyr has refused the saved API token, either because it expired or was revoked. In this situation, mabl stops syncing results for the integration. To reconnect, open the integration, enter a new API token, and click Save.

Link Zephyr test cases to mabl tests

Link mabl tests to Zephyr test cases in the Test cases section of the test information dialog. A Zephyr test case key starts with its project key, such as PROJ-T123 in the project PROJ. Plan runs record results against the linked cases.

1. On the test details page, click on the pencil icon to update test information.
2. In the Test cases section, under Add test cases, choose the Zephyr project in Add from. The section opens on the system the test already links.
3. Search the project's test cases by name or key and pick one. You can also type a full key, such as PROJ-T123, and press Enter.
4. Click Save.

Each linked case appears as a chip with the Zephyr logo and the case name. Click on the name to open the case in Zephyr. A test can link up to 20 cases, and near the limit the heading shows how many are in use, such as Linked (18 of 20).

If a key doesn't exist in Zephyr, mabl flags it below the list, and plan runs skip it until you fix or remove it. The case's project must also be one of the integration's Zephyr project keys. Otherwise, plan runs skip the link as "project not in the integration".

To remove a case link, open the test information dialog, click on the × on the case's chip, and click Save.

On the test details page, linked Zephyr keys appear beside the test's other details under the Zephyr logo. Click on a key to open the case in Zephyr.

Sync plan run results to Zephyr

When a mabl plan run syncs, whether manually or automatically, mabl adds a test execution in Zephyr for each Zephyr test case linked to a test in the plan. A plan doesn't need to be fully linked. mabl tests without a linked Zephyr test case run as usual in mabl, but they aren't recorded in Zephyr and aren't listed as skipped.

The Zephyr button in the plan run header shows whether the run's results are in Zephyr. Click on the button to check the sync status and to sync the results. If a sync doesn't record everything you expected, see When a sync is incomplete or fails.

Sync a plan run manually

1. Open the plan run output page and click on the Zephyr button.
2. Click Sync to Zephyr.
3. The button reads "Syncing to Zephyr" while the sync runs, then shows the result. The Results went to section of the popover links to the test cycle on your Jira site.

Automatic sync

With automatic sync on, mabl records every completed plan run in Zephyr without anyone clicking Sync to Zephyr, including plan runs with failures.

1. Go to Settings > Integrations and click on the pencil icon for the Zephyr integration.
2. Under Setup, check Automatically sync plan run results.
3. Click Save.

Moving forward, mabl syncs results to Zephyr after plan runs complete.

• Plan runs that completed before automatic sync was turned on are not included. To record these runs, open their results page and sync manually.
• To turn off automatic sync, uncheck Automatically sync plan run results in the integration settings and click Save. Runs that already synced stay in Zephyr, while new runs show a "not synced" status until someone syncs them manually.

What mabl records in Zephyr

For each Zephyr test case linked to a test in the plan, mabl adds one test execution to the plan run's test cycle:

• The execution's status is the one you mapped under Execution statuses.
• If a case is linked to several mabl tests, one failing test marks the execution as failed. Otherwise, one passing test marks it as passed. The execution is recorded as skipped only when every test was skipped.
• The execution records when the run ended and how long it took.
• The execution's comment starts with "Result reported by mabl" and lists each mabl test behind it, with its status and a link to its run.
• If you turned on screenshots, a failed execution carries the final application state of the failing test run.
• If you turned on Record the mabl environment and the names match, the execution carries the Zephyr environment.
• Each recorded test case gets a web link to every mabl test behind its result, named mabl test: <test name>, so you can open the mabl test from Zephyr. If someone deletes the link, the next sync adds it back.

With Create a test cycle for each plan run, mabl names the new test cycle after the plan and the time of the run, places it in the folder you set, and links it back to the plan run in mabl.

Sync again records only the cases that this plan run has no execution for yet, so syncing twice doesn't create duplicate executions. A case key fixed on the test is picked up by the next plan run.

What isn't synced

• Ad hoc runs of a single test. Only plan runs are recorded in Zephyr.
• Cancelled or stopped plan runs.
• Tests that are disabled in the plan, or that have no linked Zephyr test case. They produce no execution in Zephyr and aren't listed as skipped.
• IDs for other test case management systems. They're left for the system they belong to.

When a sync is incomplete or fails

Three statuses on the Zephyr button mean that a run needs attention:

• Zephyr — N links skipped: the run is synced, but some case links couldn't be recorded. Click on the button and review the Skipped section of the popover. See Skipped case links.
• Zephyr sync failed: the sync stopped before it finished. The popover says how many cases were recorded before it stopped. Click Sync again. If it fails the same way, check the integration settings.
• Zephyr sync status unavailable: mabl couldn't check whether this run was synced. Reload the page, or sync the run.

The plan's run history and the Plans page show the same state as Sync incomplete or Sync failed, so you can spot a run that needs attention without opening it.

Skipped case links

When a case link couldn't be recorded, the popover lists it with one of the following reasons:

Sync again only helps when the cause was on the Zephyr side. A case key that you correct on the test is picked up by the next plan run.

Create a test from a Zephyr test case

If a Zephyr test case already describes what a test should do, you can hand it to the mabl agent when building new tests. When the workspace has an enabled Zephyr integration, the + menu below the prompt on the Build tests page includes a Zephyr test case option.

1. Go to Build tests and, on the New test or Edit test tab, click + below the prompt and choose Zephyr test case.
2. If the integration covers more than one project, choose the Project.
3. Filter the list by test case name or key, and pick the cases you want. You can add up to five test cases.
4. Describe anything the test cases don't cover, such as test data or the starting state, and send the prompt.

The agent reads each test case's objective, precondition, and steps, and uses them as the specification for the test outline. Steps that call another test case are read too. If the agent can't read a test case in full, the outline says which one.

The list shows the first 5,000 test cases in a project. To find a test case beyond that, type its exact key, such as PROJ-T123.

Automatic linking

If you build a new test from exactly one Zephyr test case, mabl links the test to that case when the agent saves it so that future plan runs record results against it. If you reference several test cases when building a new mabl test, you'll need to add the links yourself in the Test cases section.