Use generative AI to review failed runs

Troubleshooting failed tests can consume a lot of time. With Auto Test Failure Analysis (Auto TFA), you can reduce the time you spend troubleshooting and devote more time to more important priorities in your schedule.

When a workspace editor or owner opens up the output for a failed test or plan run, mabl sends the output to a large language model. The model returns an analysis of why the test or plan failed. For failed test runs, the model also suggests a failure reason to help your team save time on review.

This article lists the output that mabl uses to generate failure analyses for browser and mobile tests, API tests, and plans.

A failure analysis for a plan run

Browser and mobile tests

For browser and mobile tests, mabl sends the following data to the model to analyze the failure:

• Test name, description, and steps
• Test output logs and screenshot from the failing test step. If there are multiple failing steps in a test, mabl uses the first failing step for its analysis
• Test output logs and screenshot from the same test step in a recent passing test, if there is one

API tests

For API tests, mabl sends the following data to the model to analyze the failure:

• Test name, description, and steps
• The variables defined in the test run and their values
• Information on the request and response for up to 5 failing test steps
• Information on the request and response for the same test steps in a recent passing run, if there is one
• The pre-request script, post-request script, and assertions for failing test steps

Plans

For plans, mabl sends the following data to the model to analyze the failure:

• Plan name and description
• The last updated time and last user to update
• Plan run information, including status, trigger method, and run time
• Contents of the plan, including an ordered list of stages and test runs
• AI-generated analyses and failure reasons for failed browser, mobile, and API tests in the plan run. 

For plans that ran before June 16, 2025, test failure analyses have not been automatically generated. Viewing a plan run from before this date will only analyze up to 10 failed tests from the plan run.