Troubleshooting browser tests

Through output logs, screenshots, and more, mabl makes it easy to debug failing browser tests. This article suggests an approach for reviewing the test output of a failed browser test to isolate the issue:

1. Start with screenshots
2. Review logs
3. Look upstream
4. View the last passed test
5. Replay in the mabl Trainer
6. Look for timing issues

Start with the screenshots

Whenever a test fails, mabl takes a screenshot as soon as it fails the test. The test output page highlights:

• The failed step
• A screenshot of the application on the failing step
• A brief description of the failure

Reviewing the screenshot of a failed step

Screenshots can give you the information you need to start diagnosing the failure:

• Is there a network error?
• Did the elements on the page load?
• Did an unexpected popup appear?
• If the failed step has a "visual change" label, compare the current and baseline screenshot side by side.
• Did mabl fail to connect to your application?

Sometimes the failed step may not have a screenshot or the application is still processing some action. In this case, you can review the screenshot associated with the test step labeled "Capture state of application after the test" to understand the state of the application when the test failed.

Review the logs

Click on the LOGS tab to view more in-depth detail on what happened during the step. By clicking through the tabs in the LOGS section, including mabl Activity, Network, DOM, and Console logs, you can develop an understanding of what happened at the time of test failure.

Reviewing network logs before a failed step

For more information on understanding test output logs, see the article on browser test output.

If mabl fails find the target element

mabl uses a number of intelligent find strategies to create resilient test steps that locate elements in your application. If mabl fails to locate an element during a test step, you can collect more information on the issue in the test output and the mabl Trainer.

Test output

Start by looking at the screenshot for the step:

• If the element isn't present, investigate the previous steps. The issue may have been caused earlier in the test.
• If the element is present but the step fails, check to see whether any recent cloud runs have passed. Comparing the failing step to a passing step can help you identify differences that may have caused the issue.

Click on the LOGS tab to get more detail on what happened during the step:

• mabl Activity logs: these logs document the target element's attributes and how mabl searched for the element.
• Element history: mabl tracks the element history for natively recorded steps. Click on Element history to view the current properties of the target element. If the element details or the element history target the incorrect element, retrain the step.
• DOM: mabl can only record steps on elements that exist in the DOM. Click on the DOM tab to view a DOM snapshot of the application when mabl executed the step.

Open the test in the Trainer

In addition to reviewing test output, you can also troubleshoot a find step in the mabl Trainer:

• Get information about the step: Click on the More actions menu for the step in question. Select "Show step info" to target element details, such as the element tag, innerText, and XPath.

• Replay the step: Click on the More actions menu of the step you're troubleshooting and select "Play through here." If the test starts to deviate earlier in the test, find the step where the deviation occurred and troubleshoot that step instead.

• Ensure that the element is visible during the step: If the element is obscured by another element, try changing the viewport width. If the element is visible, try adding Configure Find. And if the step already uses Configure Find, ensure that the Configure Find attributes match the target element.

Look upstream

The step that mabl fails on may not be the step that caused the issue. By investigating screenshots in prior steps, you can determine whether the test started diverging from the expected behavior before the failed step and start to isolate the issue.

As an example, consider a step that interacts with a dropdown set to "Find first." If a new dropdown is added before it, the step will find the new dropdown instead of the intended element. When that happens, mabl still interacts with the element property but fails when mabl tries to click on a dropdown option that doesn't exist.

Not sure what to look for? Try investigating prior steps with the following labels:

• "Passed with warning": this label indicates that a step had some problems during execution but still passed. Check the screenshot of the subsequent step to determine whether the "Passed with warning" step actually completed the expected action.
• "Auto-heal": if mabl is unable to find the target element, mabl looks for the strongest match to what it knows about the target element. If mabl used an incorrect element to complete the step, the test might fail in a subsequent step. Learn more about auto-heal here.

View the last passed test

If a failed test has passed previously, find the most recent passing test and compare the differences between the two runs.

For tests that have been run multiple times or across multiple browsers, click on the dropdowns for the Browser and Time of run sections on the test output page to see different runs.

Switching between different browser test runs

Start by comparing high-level test output information for passing and failing runs. For example, if you find that passing runs and failing runs consistently differ in environment, plan, or time of run, you can use that information to isolate the issue.

Replay in the mabl Trainer

In some cases, such as when mabl encounters browser pop-ups such as basic auth, the screenshot and output logs may not be enough to determine the cause of the failure. Replaying the test in the mabl Trainer is one of the best ways to get a live look at how your tests are executing.

To replay the test in the mabl Trainer, take the following steps:

1. On the test output page, click on the Edit test button.
2. In the dropdown, select Launch Trainer and replay to failed step.

Replaying a test to the failed step in the mabl Trainer

While replaying your test in the mabl Trainer, it's helpful to have Chrome DevTools open to view any messages from the console and determine whether those have any affect on your test. In addition to console errors, it can also be helpful to view the Network tab if your test is running particularly slowly.

Look for timing issues

Latency is highly variable for many applications and in test environments. If a particular test or step fails intermittently, consider whether there are intermittent spikes in latency in the test environment. Some steps you can take to investigate performance issues include:

• Check the speed index, or the perceived load speed of your application, for increases in app load time.
• Download step trace to review a timeline of the step as mabl recorded it.
• Add a wait until step before the failing step to account for spikes in latency.

Confirm that your test environment has the capacity for mabl load

mabl executes all tests within a plan in parallel by default. If your test environment is deployed on infrastructure that is not designed to handle many concurrent users, or you have unexpected failures, you can try the following steps to troubleshoot:

• Trigger tests ad hoc
• Change the plan to run tests sequentially

If your test is taking longer than expected to run

If a browser test takes longer than expected to run, see the guide on optimizing test performance to troubleshoot the issue.

Reach out to the mabl team

If you've tried the above and still can't determine the cause of the failure, feel free to reach out in-app or by email to the mabl support team whenever you have questions! We're always happy to help.