Link crawler FAQs

Commonly asked questions about the link crawler

Where can I find the link crawler?

The link crawler is a plan called "Check all pages for broken links and errors" that consists of one test: "Visit all linked pages within the app."

You can find the link crawler plan for a specific application on the Plans page. Make sure you set the "Default Plans" filter to "Include mabl-created plans":

What types of links does the link crawler check?

The link crawler only checks the links of anchor elements (<a> tags) that match the domain of the application URL and have one of the following supported content types:

• text/html
• application/xhtml+xml
• application/xml

Other HTML elements that are intended to perform page navigation on click, such as tabs that use the same URL with data-controls to change the page content, are not considered links in link crawler tests.

Can I exclude pages from the link crawler test?

No, you cannot specify pages to exclude from the link crawler test.

If you want to exclude certain URLs from page coverage metrics, you can configure an exclusion list on the page coverage dashboard. However, this setting does not stop the link crawler from checking those URLs.

Can I edit the link crawler test?

No, the test steps for the link crawler are auto-generated by mabl and cannot be changed. However, you can edit some link crawler settings, such as adding a login flow or changing the starting URL. See the article on link crawler settings for more details.

Can I delete the link crawler test?

No, you cannot delete the link crawler test. Link crawler tests are system-generated when you create an application and system-deleted when you delete the application.

By default, the link crawler plan is filtered out of the Plans page. To include the link crawler plan in the list of plans, update this "Default Plans" filter to "Include mabl-created plans."

How can I turn off the link crawler test?

If you want to stop the link crawler test from running, disable the "Check all pages for broken links" plan.

How can I configure the link crawler test to log into my app?

You can add an auto-login flow at the plan level. If the auto-login flow is unable to log into your app, try adding a custom login flow. See the article on link crawler settings for more details.

Why is the link crawler run showing a stopped status?

Mabl browser tests have a maximum run time for six hours. Link crawler tests that run in excess of six hours are automatically stopped by mabl. Test runs can also be stopped by a user in the mabl app.

Why didn't the link crawler test check all the pages in my app?

The link crawler can visit no more than 500 pages and verify no more than 10,000 unique links. If the number of links in your application exceeds these numbers, the link crawler will only visit the first 500 pages or check the first 10,000 links that it encounters.