Troubleshooting failed tests can take up a lot of time. With GenAI test failure summaries, you can reduce the time you spend troubleshooting and devote more time to other important priorities in your schedule.
When you generate a test failure summary, mabl sends test output to a large language model for analysis and returns a succinct explanation of why the test failed. This article lists the output that mabl uses to generate a test failure summary by test type.
Test failure summary for an API test
Early access feature
GenAI test failure summaries are currently available as an early access program. Prior to general availability, changes to the feature may be made without notice.
Browser and mobile tests
For browser and mobile tests, mabl sends the following data to the model to generate a failure summary:
- Test name, description, and steps
- Test output logs and screenshot from the failing test step
- Test output logs and screenshot from the same test step in a recent passing test, if there is one
API tests
For API tests, mabl sends the following data to the model to generate a failure summary:
- Test name, description, and steps
- The variables defined in the test run and their values
- Information on the request and response for up to 5 failing test steps
- Information on the request and response for the same test steps in a recent passing run, if there is one
- The pre-request script, post-request script, and assertions for failing test steps