Restricting the crawler to a specific application

You can add an optional "Use strict URL prefix" parameter to your Link Crawler plan to restrict the URLs that it visits. With this parameter set to "true", mabl will only crawl pages with a URL prefix that matches the base URL for your application.

To illustrate, if you have an application with a base URL of https://app.example.com, the crawler will visit any linked pages that include example.com as the base domain, including www.example.com and docs.example.com. With this parameter enabled, mabl would only crawl pages that begin with https://app.example.com. We could further restrict the pages that mabl visits by including sub-pages in the URL for the plan. Thus, if the plan was set to test https://app.example.com/app1/ and this parameter was set to true, mabl would only visit linked pages that begin with https://app.example.com/app1/.

Note: More information on the default behavior of the crawler is available here (https://help.mabl.com/v1.0/docs/about-the-link-crawler)

Restricting the crawler to a specific URL prefix

  1. Open the plan that includes the "Visit all linked pages within the app..." journey.
  2. Examine the page for the "Edit" button.
  3. Click "Edit"
  1. Scroll down to the "Journeys" section.
  2. Below the "Visit all linked pages..." journey, click "Add Parameter"
  3. Choose "mabl parameter" as the Parameter Type
  4. Choose "use strict URL prefix" as the Parameter Name
  5. Choose "true" as the Parameter Value

Restricting the crawler to a specific application