Xray results reporting

With Xray results reporting, your team can publish an up-to-date record of your mabl testing results directly into an Xray project. For teams in more regulated industries, this integration helps you ensure compliance with industry standards, document key evidence for audits, and provide better visibility into the testing process.

This article explains how to set up Xray results reporting in mabl.

• Set up the integration
• Associate mabl tests and plans with Xray
• Publish results to Xray

Set up the integration

Gather your Xray credentials

To connect mabl with Xray, you’ll need to copy and save the necessary credentials from Xray. You'll need the following for later setup steps in mabl: 

• Client ID
• Client secret

To obtain these credentials, a Jira admin for your Xray project must first create an API key. The API key must be associated with an Xray user that has sufficient permissions to browse Xray projects and create, edit, and link issues within the relevant Xray project.

Add the integration in mabl

In the mabl app, go to the integrations page: Settings > Integrations. In the Available integrations section, click on the + Setup button for Xray sync.

On the integration setup page, provide the following connection details:

• Name - a descriptive name for your integration for the mabl side
• Xray client ID - the client ID from your Xray credentials
• Xray client secret - the client secret from your Xray credentials
• Project key - the unique, shorthand identifier of your project. For example, if issues are "PROJ-789" and "PROJ-790", your project key is "PROJ".

Select which results to sync

At a minimum, mabl will report run metadata, including the status (pass/fail) and execution start/end times. In the Result synchronization section, you can configure the integration to report additional details:

• Sync failed steps only - indicate whether you want mabl to send all results or only results for failed steps.
• Map status and evidence for steps - sync mabl test step artifacts to Xray step-level evidence. This configuration requires additional setup steps. See the article on Xray step-level sync for more details.
• Artifact types - specify which step-level artifacts should be sent to Xray

When you save the integration, you can start associating tests with your Xray project.

Note: Changes made to the integration's sync settings, such as toggling artifact types, only apply to syncs created after the change. To update artifacts for previously synced results, resync the results from the mabl test or plan run output page after the integration settings have been updated.

Associate mabl plans and tests with Xray

After setting up the integration, you can start associating mabl plans and tests with your Xray project. The following table outlines how mabl entities map to Xray entities:

Strict naming required

To successfully sync results, your Xray project must use the exact issue type names "Test Plan" and "Test". If these names differ in your Jira instance, they will not be available for association in mabl.

To verify, hover over the issue icon in your Xray project. The tooltip must explicitly display "Test Plan" or "Test" as the issue type for the sync to work.

Recommended workflow for syncing results

To get the complete results for an associated mabl plan in Xray, you must associate both the mabl plan and every single test within the mabl plan with a corresponding Xray entity in the same project. If a mabl test isn't associated, its results will be omitted from the Xray Test Execution report.

To ensure full coverage in Xray, we recommend this two-step association workflow:

1. Associate a mabl plan with an Xray Test Plan
2. Associate all mabl tests within the plan with their own Xray Test Case in the same project

Associate plans

To associate a mabl plan with an Xray Test Plan, click on the Xray button on the new/edit plan page. Enter the Xray project and Xray Test Plan ID.

Associate tests

To associate an existing mabl test with an Xray Test Case, go to the test details page in mabl and click on the pencil icon to update info.

In the Update test information modal, click on the Xray button and enter the Xray project and Xray Test Case ID. Note that the Custom test case management field is intended for configuring a custom test case management integration and should not be used for Xray results reporting.

Publish results

After associated mabl tests and plans finish running, you can publish the results to Xray as a Test Execution. On the output page for a test run or plan run in mabl, click on the refresh icon next to the Xray ID.

Plan runs

When you sync a plan run, results are published to the Xray Test Plan as a new Test Execution. A Test Execution for a mabl plan run only includes results for mabl tests that are linked to an Xray Test Case within that same project. Any tests in the mabl plan that aren't associated with the Xray project are excluded from the Test Execution report.

For example, consider a mabl plan that is associated with an Xray Test Plan and executes 5 tests:

• If 3 of the tests are linked to an Xray test case, the results of those tests are published in the Test Execution report.
• If 2 of the tests are not linked to an Xray test case, their results are excluded from Xray and only visible in mabl.

Test runs

If you are troubleshooting an issue in a single test, you can sync the results from the test run output page. Results are published as a new Test Execution associated with the Test Case ID.