When a plan executes in the cloud, mabl schedules and runs the tests according to the plan’s configured settings. You can review the output of a plan run on the plan details page in the app.
This article explains key concepts related to plan run output.
The plan details page
Other ways to review plans
Besides reviewing plan run output in the mabl app, you can also review plan runs by integrating them into your existing workflow:
- Set up the Slack or Teams integration to notify you of plan run statuses.
- Pipe data from the plan run summaries API endpoint into an internal reporting tool.
- Set up the BigQuery integration to send data from plan runs upon completion.
Status
Plan runs have one of the following statuses:
Running
A “running” status indicates that the plan run has started and there is at least one test in the plan that hasn’t finished running.
Passed
A “passed” status indicates that all the tests have passed.
Failed
A “failed” status indicates that one or more tests have failed.
By default, a failed test stops any subsequent tests in the plan from starting. These tests show a “skipped” status. To change the plan’s default behavior on failure, update the plan stage settings.
Stopped
A “stopped” status indicates that the plan couldn’t run all tests for a reason other than failure. Reasons for a stopped plan run include:
- A workspace member stopped the plan run
- A workspace member stopped a test in the plan
- One or more tests in the plan run exceeded the maximum test run time
For more information on test run statuses, click here.
Duration
The duration, or run time, of a plan run measures the time it takes from triggering a plan run to the moment when all tests are completed. In some cases, tests don’t start running immediately. For this reason, you may notice a difference between the total run time of tests in a plan and the run time of plan.
Triggers
The plan details page also lists how each run was triggered and whether it was a retry of failed tests from a previous run.
Manual
Ad hoc plan runs show the name of the workspace member who manually triggered them.
A manually triggered plan run
Schedule
Runs triggered “by schedule” indicate that the plan is configured to run on a set interval or on specific days and times. See scheduling tests for more details.
API
Runs triggered “by API” indicate that the plan was triggered as part of a mabl deployment event.
mabl local
A plan run with a "mabl local" trigger indicates that the tests associated with the plan were triggered locally from the mabl CLI with the --from-plan-id option and the --reporter option set to mabl. See Unified Reporting for more details.
Playwright
A plan run triggered by "Playwright" indicates that the tests were triggered in Playwright and published to the mabl app with the Playwright reporter package.
Plan run IDs
For every plan run, mabl generates a unique ID ending in -pr. To obtain a plan run history that includes plan run IDs, click on the Download CSV button on the plan details page.
CSV output is limited to 200 rows. If you require more data, consider using our BigQuery Feed Export Feature.