We're happy to share that GenAI Test Failure Summaries are now generally available for browser, mobile, and API tests!
Troubleshooting failed tests takes time. With GenAI Test Failure Summaries, you can quickly understand failures, identify patterns and trends, and reduce overall time spent on identifying and resolving bugs.
Advanced AI add-on
GenAI Test Failure Summaries are available as an add-on to the mabl core subscription. Reach out to your customer success manager to learn more.
How it works
If your workspace includes the Advanced AI add-on, when you open up the test output for a failed test run, mabl sends the output to a large language model for analysis and returns a succinct explanation of why the test failed.
The failure summary appears in the purple box
Expand the summary to view more details about why the test failed, including follow-up actions:
- Assign a failure reason - when applied consistently and accurately to failed test runs, failure reasons can help your team understand overall application quality and save time on review.
- Regenerate the failure summary - click Regenerate to have the model reanalyze the failure and provide a new summary.
- Give feedback - use the thumbs up and thumbs down icons to give us feedback on the quality of the failure summary.
An expanded failure summary
What's changed since EA
Since introducing GenAI Test Failure Summaries back in August, we made the following enhancements:
- Expanded support to include API tests
- Added support for failure summaries in Japanese
- Improved the accuracy of the suggested failure category
- For network issues, network logs are included to improve the depth of the summary
- Updated the failure summary analysis to appear inline with the test output instead of as a popover element.
Learn more
To learn more about the test output data that mabl uses to generate test failure summaries, check out the docs.