Plan FAQs

This guide addresses some frequently asked questions on:

Creating & configuring plans

Do I need to create plans to run my tests?

If you want to do any of the following…

… then you need to create plans.

What is a stage?

A plan stage is a grouping mechanism that helps you specify the order in which tests will run in a plan. Plan stages can be useful for test runs that require setup and/or teardown in an environment.

Do I always need to put a test in a plan?

No. It is not necessary for a test to be associated with a plan. When you create a new test, you can select "No plan" in the Add to plan section. You can trigger ad hoc test runs locally and in the cloud.

If you change your mind later, you can always add the test to a new or existing plan:

Create a new plan/edit an existing plan
Add the test to the Tests section.

Can I use multiple data tables with the same plan?

Yes, see this guide for more information.

Can I use more than one set of credentials on a plan?

You can only assign one set of credentials to a plan. If you need to use multiple sets of credentials for a plan, you could consider an alternative strategy. For example:

Environment variables: This option is suitable for teams who use different sets of credentials in different environments.
Designated login flows: This option is suitable for teams who use several sets of credentials in an application/environment. The flow for each credential/user role can be reused across different tests.

Default mabl plans

What are the 'Verify home page load and login' and 'Check all pages for broken links and errors' plans?

When you configure a new application in mabl, with the goal of ensuring you get some baseline coverage quickly, we automatically add two tests and plans that run for free to check that your application is up and running as well as let you know of any potentially broken links.

The "Verify home page load and login" plan contains one test: "Visit home page." This a basic smoke test that visits the base URL of your environment and logs in, if applicable, with an auto login flow. It runs every 4 hours and on deployment.
The "Check all pages for broken links and errors" plan also contains one test: "Visit all linked pages within the app." This test is also known as the link crawler test. It runs once a week.

If you do not want these plans to run, you can toggle them off.

Can I edit the tests in the default mabl plans?

You cannot edit the tests in the default mabl plans. You can edit some settings at the plan level: environment, device settings (desktop or mobile), and login credentials.

If you want to add a custom login flow to a default mabl test, go to the Test Details page and take the following three steps:

Click on the Copy with Custom Login button.
Select the login flow that you would like to use.
Click Duplicate. A copy of the default test with the custom login flow will be created.

📘
Copying default tests
Cloud runs of a copied default test with the custom login flow will count against your monthly quota.

Can I delete the default plans that mabl added?

Default plans cannot be deleted manually. The delete icon is disabled for these plans.

If you do not want these plans to run, you can toggle the plan off.
When you delete an application from your workspace, its associated home page and link crawler plans will be deleted automatically.

Running plans

Can I run plans locally?

It is not possible to trigger a local plan run from either the mabl Desktop App or the mabl CLI.

In the mabl CLI, you can trigger local test runs with the --from-plan-id option: mabl tests run –from-plan-id <plan_id>. It will trigger a local run of all the tests in a plan sequentially, in alphabetical order according to the test name. This is not considered a plan run, since the settings that are configured at the plan level, such as concurrency, stages, DataTable scenarios, and shared variables, are not used.

Can I run a plan against multiple environments?

Yes! You can configure this in the plan settings.

Can I run a plan in only one of the plans configured to it?

If your plan is configured to run in multiple environments, but you only want it to run in one of the environments, there are two options:

Create a duplicate plan using the top navigation actions. In the copied plan, select the one environment that you want to use. Give this plan a helpful name for distinguishing it from the original, multi-environment plan.
Trigger a mabl deployment event to run that plan in one environment.

Can I run tests in a plan sequentially?

Yes! You can configure tests to run sequentially in a plan stage. See our documentation on plan stages for more information.

Can I limit the number of tests that run at one time?

Yes! You can set a concurrency limit to control the number of tests that run at any given time.

Can I trigger more than one plan to run at a time?

Yes! You can set up a mabl deployment event to trigger multiple plans at the same time.

Can I run a plan on a branch?

Yes! See our documentation on running plans on branches for more information.

Can one test know about the values generated in another test?

Yes! You can configure a plan to share variables between tests.

Can I use the same Mabl mailbox address across multiple plan runs?

Mailbox email can only be received (asserted) in the same test that generated the address

Reviewing plan runs

Can I export the run history of a plan?

Yes, you can export plan run history as a CSV file. Go to the Plan Details page for the plan and click on the Download CSV button.

Is there an API endpoint to see the results of plan runs?

There are three ways to retrieve the results of plan runs:

Execution Result API: Get the results of plans triggered by a mabl deployment event in JSON format
Post-execution webhook: Get the results of all tests triggered in a workspace in JSON format
CI/CD Integration: If you use the Jenkins, Bamboo, or Circle CI integration with your mabl tests, you can get the results of your test runs in a JUnit report.

How is plan run time calculated?

Plan run time measures the time it took from when a plan was scheduled to when it was completed.

Updated 7 months ago