When a plan run comes back with several failures, it takes time to tell whether you're looking at one systemic problem or fifty separate ones. With automatic failure categorization, the mabl agent evaluates each failed test run and assigns a failure reason for you, so you can get a quick gist of what went wrong without having to categorize every failure by hand.
Outside the app, you can access auto-assigned categories from test run summaries in the mabl API and the _run_categorization BigQuery export table. Use these tools to build robust dashboards to follow trends in failure types.
How to get access
Automatic failure categorization is opt-in for existing workspaces and off by default, so your current categorization and reporting are unaffected until you choose to enable it. New workspaces created after June 17, 2026 have it on by default.
Workspace owners can turn on automatic failure categorization in agent settings:
- Go to Agents > Settings
- Click on the pencil to edit Agent behavior.
- Turn on automatic failure categorization.
How it works
After enabling automatic failure categorization, every failed cloud test run is evaluated and assigned a failure reason automatically. Depending on the type of failure, the mabl agent may choose from a built-in failure reason or one of your team's custom failure reasons. To steer which custom reason the agent picks, give each one agent guidelines that describe when it applies.
Manual assignments
Failures that you categorize manually are never overwritten, and you can recategorize an auto-assigned reason at any time. Manual assignments always take precedence.
Learn more
To learn how the mabl agent investigates failures and suggests reasons, see Conversational results analysis. For more on classifying failures, see Failure reasons.