We're happy to share three improvements to mabl's results analysis for failed runs:
- Richer supporting evidence - the initial analysis in the Results summary view now surfaces more detailed, structured evidence, including screenshots, historical trend charts, log snippets, and step-level detail.
- Agent summary for deployment failures - when a deployment finishes with failures, the agent aggregates findings across all the plan runs triggered by the deployment, giving you a single view of what went wrong instead of investigating each plan run separately.
- Evidence details available in API, BigQuery, and MCP - surface detailed failure analyses in your custom reporting workflows.
Richer supporting evidence
Previously, the Supporting evidence in the Results summary view provided a few initial artifacts, such as screenshots and run history. With this update, the agent generates more robust supporting evidence as part of the initial failure analysis, so the full picture is on the page the moment you open a failed run.
Agent analysis for deployment failures
This release extends failure analysis to deployment events alongside the existing test run and plan run analysis. When a deployment finishes with failures, the agent aggregates findings across all the plan runs the deployment triggered, so you get a single view of what went wrong instead of investigating each plan run separately.
Evidence details available in the API, MCP, and BigQuery
The saved analysis is now available outside the mabl app, too:
-
Reporting API: the
failure_analysisfield on test run, plan run, and deployment event endpoints now includes the full evidence markdown -evidence_details- alongside the synopsis and summary. -
BigQuery export: a new
*_failure_analysistable in customer BigQuery datasets makes the saved analysis available for trend reporting and cross-run analytics. -
mabl MCP: the
#analyze_failuretool now returns the agent's full analysis on demand, so you can investigate specific failures from your AI client with the same depth as the Results summary page.
Failure analysis is enabled for workspaces with mabl's GenAI features turned on. If you don't see analysis on your failed runs yet, reach out to your customer success manager.
Check it out
Open any failed test run, plan run, or deployment event in the mabl app and head to the Results summary tab. The agent analysis renders the structured evidence directly on the page. Use the conversational chat to dig into specific patterns, compare runs, or generate a custom PDF report.
To pull failure analysis into your own systems:
-
API: Add
failure_analysis.evidence_detailsto your queries againstGET /results/testRun/{test_run_id},GET /results/planRun/{plan_run_id}, orGET /execution/result/event/{event_id}. -
BigQuery: Query the new
*_failure_analysistable in your exported dataset. See the BigQuery integration docs for the full schema. -
MCP: Ask your AI client about a recent failure (for example, "Why did this deployment fail?"). The client invokes the
#analyze_failuretool and pulls the agent's findings into the conversation.