As part of our ongoing efforts to improve plan run options, we have updated the logic for plan reruns. If a test in the original plan run executed multiple times against different URLs, mobile devices, browsers, or DataTable scenarios, a plan rerun will only execute the tests with the particular combination of parameters that failed in the original plan run. Previously, plan reruns would schedule all copies of the original test to run.
This change makes plan reruns a more effective method for diagnosing test issues and reduces overall test credit usage.
For example, the below plan run scheduled the same test against Chrome and Firefox, and then was re-run with "Rerun only failed tests in this plan" selected. Previously, this re-run would have contained both a Chrome and Firefox run. Now, the re-run contains only the failed Firefox run.
Example of a plan rerun which only schedules the original failed test runs
This updated logic applies to:
- Automatic reruns with “Retry failed tests only” selected
- Manual plan reruns with “Rerun only failed tests in this plan” or Rerun failed and skipped tests” selected
This change does not affect the run multiplier. If a plan is set with a run multiplier, reruns will still generate each new test run multiple times according to the value of the run multiplier.
To learn more about plans within mabl, click here.