We’re happy to introduce test failure summaries, a new way to get insight into your mabl cloud runs!
Now available in early access for browser tests, test failure summaries use generative AI to analyze test output and provide a succinct explanation of why the test failed. With test failure summaries, you can quickly understand failures, identify patterns in failures, and reduce overall time spent on identifying and resolving bugs.
Test failure summaries are currently available for all mabl customers and trial users as an early access program. Prior to general availability, changes to the feature may be made without notice.
Try it out
Before you start, update to the latest version of the mabl Desktop App: Help > Check for updates. During the early access period, workspace owners can enable test failure summaries on the Labs page: Settings > Labs.
Generating failure summaries
On the test output page for a failing browser test, click on the Why did this test fail? button at the top of the page to generate a failure summary.
Test failure summary
Mabl sends the following test output data to the model to generate a failure summary:
- Test name and description
- Test output logs and screenshot from the failing test step
- Test output logs and screenshot from the same test step in a recent passing test, if there is one
If you’re satisfied with the generated failure summary, click on the Accept button to save it. Once saved, you can access the failure summary from the test output page whenever you need to.
Suggested failure reasons
As part of the failure summary, mabl includes a suggested failure reason. Your workspace's categorized failure reasons appear on the mabl dashboard, the release coverage dashboard, and in the BigQuery integration. When applied consistently and accurately to failed test runs, failure reasons can help your team understand overall application quality and save time on review.
Categorized failures on the mabl dashboard
Submit feedback
Since test failure summaries are available as an early access program, we highly encourage you to provide feedback on this feature in the mabl Product Portal. Your feedback helps us build a better product for you and our community!
Test failure summaries are built on top of Google Cloud's AI tools. Neither mabl nor our service partner, Google Cloud, use customer data for training these models. If you have any concerns about the use of generative AI in mabl, please reach out to your customer success manager.
For more information about how mabl handles customer data, see the article on data encryption.