Debugging failed tests - Best Practices

Through output logs, screenshots, and more, mabl makes it easy to debug your failing tests.

Start with the screenshots

Whenever a test fails, mabl will take a screenshot as soon as it fails the test. When you open the output logs for the failed test, you'll be taken to the failed step and shown the failure screenshot on the right. You'll also see a brief description of the failure which you can use before diving deeper into the logs.

By examining that screenshot, even just skimming it, can tell you a great deal about your failed test. Is there a 404 error, is there a popup that mabl hit, did the elements on the page load? Many times, the screenshot will give you all the information you need to know to diagnose the failure quickly and easily.

Understand what strategies mabl tried to find the element

The mabl output logs break down every strategy mabl tried, successfully or unsuccessfully, to find the element it needs to interact with. When you expand a step in the output logs, you'll see these strategies listed below with a white (i) to their left. It's also helpful to click the small clock icon in the top left of the output logs, which will switch the timestamp from server time to time since the start of the run.

Using these together is incredibly helpful for debugging, as you can see exactly how long it took mabl to find, or not find, the element in question. Upon failure, mabl will also display the exact information it used to find the element, such as: class, id, XPath, href, etc.

View the last passed test

If a failed test has passed previously, a "view last passed test" message will appear under the failed status on the output page. Clicking this link, or preferably opening it in a new tab, will take you to the last passing status. From here, it's easy to compare differences between the two runs.

Look upstream

The step that mabl fails on may not always be the step that caused the issue. For example, if the step looking for a dropdown is set to "Find first" element and a new dropdown is added before it, the step will find that dropdown instead of the intended element. When that happens, mabl will still interact with the element properly, but will fail when mabl tries to click on a dropdown item that doesn't exist. This approach is especially helpful to determine the broader context in which the error occurred.

Recreate on your own

In some cases, such as when mabl encounters browser pop-ups such as basic auth, the screenshot and output logs may not be enough to determine the cause of the failure. Using the "Edit test" button at the top of the Test Output page is one of the best ways to get a live look at how your tests are executing. Use the play button, located at the top of the trainer window to play through each step. You'll be able to see exactly what issues mabl has when running your test right here in the mabl Trainer.

If your tests are particularly long, containing many assertions and similar, it may be helpful to use the arrow keys to step through your test. Just hover over the trainer window and press the up and down keys to move through the output without having to play each step.

Open Chrome Dev Tools

While you're editing your test to look for the cause of the failure, it's helpful to have Chrome dev tools open in addition to the mabl trainer window. It's easy to view any messages from the console and determine whether those have any affect on your test. In addition to console errors, it can also be helpful to view the page load time (under the Network tab) if your test is running particularly slowly.

Link crawler failures

Reach out to the mabl team

If you've tried the above and still can't determine the cause of the failure, feel free to reach out (in-app or by email) to the mabl team whenever you have questions! We're always happy to help.