About the Link Crawler

mabl includes a "link crawler" plan and a corresponding journey to "visit all linked pages within the app" by default. The plan is configured to run against all of your application environments every 24 hours and when you notify mabl of a new deployment via the deployment events API.

If you opted the include credentials to access your application when you added the application, the link crawler will use those credentials to automatically log in to your application each time it runs at test.

You can change the configuration of the default link crawler plan(s) as you would any other plan.

When running the link crawler, mabl begins by loading the page at the base URL of your environment. After collecting information about that page (including a screen shot load time, JavaScript error information, and more), mabl will check the first hyperlink on that page. If the hyperlink is within the domain of your base URL, mabl will visit that page and collect the information noted above. mabl will then proceed to visit the second link on the original page, and then the third, and so forth. mabl will proceed to visit all of the linked pages in your application in this manner.

mabl visits all linked pages within the domain by default

With no configuration, mabl will visit all linked pages within your domain. That means that if you configure an environment with URL https://app.example.com, mabl will visit not only pages within https://app.example.com (such as https://app.example.com/page1) but also any other pages that are linked to example.com within your app. Often, this results in mabl crawling documentation sites (https://docs.example.com) and marketing sites (https://www.example.com).

About the Link Crawler