Restricting the link crawler to a specific application

You can add an optional "Use strict URL prefix" parameter to your Link Crawler plan to restrict the URLs that it visits. With this parameter set to "true", mabl will only crawl pages with a URL prefix that matches the base URL for your application.

To illustrate, if you have an application with a base URL of https://app.example.com, the link crawler will visit any linked pages that include example.com as the base domain, including www.example.com and docs.example.com. With this parameter enabled, mabl would only crawl pages that begin with https://app.example.com. We could further restrict the pages that mabl visits by including sub-pages in the URL for the plan. Thus, if the plan was set to test https://app.example.com/app1/ and this parameter was set to true, mabl would only visit linked pages that begin with https://app.example.com/app1/.

Note: More information on the default behavior of the crawler is available here

📘

Note

More information on the default behavior of the link crawler is available here.

Restricting the crawler to a specific URL prefix

1. Open the Plan Details page for "Check all pages for broken links and errors."
2. Click the "Edit" button.

4. Scroll down to the "Tests" section.
5. Click on the "Configure data-driven variables" icon for the "Visit all linked pages within the app" test.

6. Click "Add Parameter" and (if applicable) choose "mabl parameter" as the Parameter Type

7. Choose "use strict URL prefix" as the Parameter Name
8. Choose "true" as the Parameter Value
9. Click "Done" and save the plan