When testing complex applications with many parts, it's important to see the big picture. In mabl, failure reasons sort your test failures by issue so you identify patterns and adopt a more streamlined approach to resolving issues in your app.
Failure reasons are available everywhere your test result data already lives, including on the results page, the coverage overview dashboard, CSV exports, test run summaries from the mabl API, and in the _run_categorization BigQuery export table.
Assigning failure reasons
There are two ways to assign failure reasons:
- Automatically - the mabl agent evaluates the test run and assigns an appropriate failure reason based on the failure analysis. Workspace owners may enable or disable automatic failure categorization from the agent settings page: Agents > Settings.
- Manually - a member in your workspace assigns a failure reason based on human review. You can manually recategorize an automatically assigned reason at any time. Failure reasons that you assign manually are never overwritten.
Types of failure reasons
mabl supports a set of default failure reasons as well as custom failure reasons.
Default failure reasons
| Failure reason | Description |
|---|---|
| Regression | mabl has caught a bug that caused your test to fail, such as a button disappearing after a recent release or a popup appearing where it shouldn't have previously. |
| Environment issue | Any failures caused by something local to your testing, development, or other environments, such as dev credentials that are no longer valid or an environment suddenly becoming private. |
| Network issue | Any failures that may be related to mabl failing to connect to your app. |
| Test implementation issue | Failures related to how the test was originally trained. For example, a failed test with a test implementation issue could be caused by recording steps in the wrong order or accidentally deleting an important step. |
| Timing issue | Failures related to the performance of your application, such as an application failing to load an element on the page in time for mabl to interact with it. |
| Accessibility issue | The test run did not meet the criteria of an accessibility check. |
| Performance issue | Tests that failed because of performance issues in the testing environment or performance tests that failed because they met the failure criteria as defined by the test. |
| Other issue | Anything that might not fit into the reasons above. |
| mabl issue | Any failures related to how mabl is executing your tests. If you believe a test failed due to a mabl issue, we recommend also reaching out to the mabl support team. |
Custom failure reasons
Custom failure reasons capture failure modes specific to how your team works — an expired test credential, a third-party outage, a stale fixture — instead of forcing them into a broad default like "Environment issue." Workspace owners manage them at Configuration > Failure reasons in the mabl app.
A custom failure reason has three fields. Write the display name and description for your teammates; write the agent guidelines for the agent.
| Field | What it's for |
|---|---|
| Display name (up to 100 characters) | The short label your team sees in the failure reason picker and on run results, such as "Third-party vendor outage." The agent also reads it as the category's name. |
| Description (up to 200 characters) | A one-line summary shown in the picker. The agent reads it as a short gloss of what the category means. |
| Agent guidelines (up to 1,000 characters) | The instructions that tell the agent when to assign this reason. These guidelines steer automatic categorization. For more information, see writing agent guidelines. |
When managing custom failure reasons:
- Rename a failure reason by changing its display name at any time. Historical references update automatically.
- Disable a failure reason to discontinue it. Disabling hides it from the picker and stops the agent from assigning it automatically, while preserving the data already classified with it.
A workspace can have up to 50 active custom failure reasons.
mabl's default failure reasons remain available and can't be renamed or deleted.