When mabl syncs a plan run to Zephyr, what does it write to your test cycle, and what does it leave alone? This article explains where each result goes, what an execution contains, and what happens when you sync a plan run again. To set up the integration, see Zephyr integration.
Where a result goes
mabl records results in Zephyr test cycles. For a result to appear in Zephyr, the mabl test must be linked to a Zephyr test case and run as part of a plan. Depending on your integration settings, mabl either creates a new test cycle for each plan run or records every plan's results in one existing test cycle per project. You can also assign a plan to a specific test cycle in its settings, and that plan's results go there instead.
Once mabl knows which test cycle to use, it looks for the case in that cycle. If your team planned a row for it, such as a row in the Not Executed status, mabl records the result on that row and keeps its assignee and any environment set on it. Later runs record on the same row, so its history lists every run. If there's no planned row, mabl adds a new execution.
Some rows are left alone:
- Results recorded by hand: mabl doesn't change a result someone recorded by hand. If a case has only a manual result in the cycle, mabl adds its own execution next to it.
- New case versions: if a case gets a new version in Zephyr, mabl keeps recording on its existing row for the older version, replacing its earlier result there, unless your team planned a row for the new version.
What an execution contains
Each execution that mabl records includes the following:
- Status: the status you mapped under Execution statuses. If a case is linked to several mabl tests, one failing test marks the execution as failed. Otherwise, one passing test marks it as passed. The execution is recorded as skipped only when every test was skipped.
- Comment: the outcome, the mabl test linked to its run, and the plan linked to the plan run. When several mabl tests cover the case, the comment lists each test with its outcome. For a failure, the comment also shows the step and flow where the test failed, the error, and mabl's failure analysis, labeled as automated.
- Screenshot: if you turned on screenshots, a failed execution carries the final application state of the failing test run.
- Timing: when the last of the case's runs ended, and how long the run behind the status took.
- Environment: if you turned on Record the mabl environment and the names match, the Zephyr environment.
Each recorded test case also gets a web link to every mabl test behind its result, named mabl test: {test name}, so you can open the mabl test from Zephyr. If someone deletes the link, the next sync adds it back.
Syncing a plan run again
Sync again records only the cases that the plan run has no execution for yet, so syncing twice doesn't create duplicate executions. In an existing test cycle, syncing an older plan run leaves alone a row that already holds a newer mabl result. If you fix a case key on a test, the next plan run picks it up.