Skip to main content

Triggering tests from the mabl CLI

Eva Ferreira
  • Updated

In the mabl CLI, you can trigger tests to run on your local machine or integrate test execution directly into your CI/CD process. The mabl CLI is not an execution environment, but rather a tool that you can use to trigger mabl tests in the cloud or in a local/CI environment.

The following commands trigger test runs in the cloud, where parallel execution is supported:

Local and self-hosted CI environments support sequential execution only. To run tests in local and CI environments, use these commands:

Read on to learn more about how to trigger tests from the mabl CLI.

mabl deployments create

Use the mabl deployments create command to trigger a parallel execution in the mabl cloud of all plans that match a specified set of filters. Filters can include application, environment, and plan labels.

Because deployment events can execute many tests in parallel, this command is ideal for getting robust validation on a large set of tests.

Options

The mabl deployments create command supports a large number of options for configuring overrides, branches, browser settings, output, and more. To see the full list of options, run mabl deployments create -h.

To get started, use the following options to specify which plan(s) to run:

Option Description
--application-id and/or --environment-id Use at least one of these options to specify which plans you want to run.
--plan-labels Optional: run plans that match any label in the space-delimited list of plan labels.

Output

By default, the CLI outputs a link to view the results of the deployment event in the mabl app.

Examples

# Trigger a deployment of all plans with a "regression" label
mabl deployments create -e [environment-id] --labels regression

# Use a URL override to run plans against a preview environment
mabl deployments create -e [environment-id] --app-url [url-override]

mabl tests run-cloud

Use the mabl tests run-cloud command to trigger one or more tests to run in parallel in the mabl cloud.

Because cloud runs collect more detailed diagnostics than local or CI runs, this command is ideal for investigating test failures in more detail.

Options

For a full list of options, including browser and branch settings, run mabl tests run-cloud -h. At a minimum, use one of the following options to specify which test(s) to run:

Option Description
--id Run the test that matches this ID
--labels Run any tests that match the space-delimited list of labels

Output

The mabl CLI outputs a link to view test output in the app.

Examples

# Trigger a cloud run of a test that failed locally
mabl tests run-cloud --id [test-id]

# Trigger cloud runs of tests that failed in a CI build
mabl tests run-cloud --labels follow-up

mabl tests run

Use the mabl tests run command to trigger one or more browser or API tests to run on your local machine or in a CI environment. The mabl CLI returns quick pass/fail feedback without consuming your cloud credits.

Because local and CI runs give quick feedback, this command is ideal for rapid iteration and debugging on individual tests during the development phase of your pipeline. Sequential execution makes this command less efficient for larger test suites.

Options

To view the full list of options for this command, run mabl tests run -h in the CLI. At a minimum, use one of the following options to specify which test(s) to run:

Option Description
--id Run the test that matches this ID
--run-id ID of the test run to pull configuration from
--labels Run any tests that match the space-delimited list of labels
--exclude-labels Exclude any tests that match the space-delimited list of labels. Use in conjunction with the --labels option.
--from-plan-id Run tests associated with a specific plan. Note that this is not the same as a plan run in the cloud: Advanced plan configurations, such as stages, browser settings or shared variables, are not applied. Tests run sequentially in the same order they appear on the plan details page in the mabl app. All tests use the same base URL, which is the URL associated with the plan’s application and environment. If the plan is configured with multiple URLs, the mabl CLI will use one of those URLs. If you include the –url option, all tests in the group use this URL.

Using --from-plan-id
Because the --from-plan-id option does not apply advanced plan configurations or parallel execution, we recommend creating designated plans for local runs that only contain a single stage. The tests in designated local plans should be of only one type - browser or API -since all tests use the same base URL.

Output

By default, the mabl CLI only outputs results directly in the terminal. To publish results of test output to the mabl app, include the --reporter option with the argument mabl:

mabl tests run --id [test-id] --environment-id [env-id] --application-id [app-id] --reporter mabl

For more information on publishing local run results to mabl, check out the article on the Unified Reporter.

Examples

# Trigger a headless local run
mabl tests run --id [test-id] --headless true

# Trigger a local run for tests that match labels
mabl tests run --labels smoke needs-review

# Trigger local test runs associated with a plan
mabl tests run --from-plan-id [plan-id]

Browser support

Browser test runs use the Chrome browser installed on your machine.

To use a different chromium browser, update the browser path using the command mabl config set browser.path [path-to-browser]. Keep in mind that incorrectly configuring the browser path may break local test execution.

mabl tests run-mobile

Use mabl tests run-mobile to run one or more mobile tests on your local machine. The mabl CLI outputs quick pass/fail feedback without consuming your cloud credits.

Because local runs execute quickly, this command is ideal for getting quick feedback on mobile-specific functionality during development. Sequential execution makes this command less efficient for larger test suites.

Setup

Running mobile tests locally from the mabl CLI requires the following setup:

  1. Run mabl config install mobile-tools to download and install the necessary software components for interacting with your Android and iOS apps.
  2. Download the mobile build file to your local machine.

Options

For a full list of options, run mabl tests run-mobile -h. Use one of the following options to specify which mobile test(s) to run:

Option Description
--id Run the mobile test that matches this ID
--labels Run any mobile tests that match the space-delimited list of labels

Output

The mabl CLI outputs test output directly in the terminal.

Examples

# Run an Android test
mabl tests run-mobile --id [test-id] --app-file [relative-path-to-file] --platform android

# Run a set of iOS tests associated with the label "wip"
mabl tests run-mobile --labels wip --app-file [relative-path-to-file] --platform ios

Related articles