In the mabl CLI, you can trigger tests to run on your local machine or integrate test execution directly into your CI/CD process. The mabl CLI is not an execution environment, but rather a tool that you can use to trigger mabl tests in the cloud or in a local/CI environment.
The following commands trigger test runs in the cloud, where parallel execution is supported:
- mabl deployments create - triggers a deployment event, which runs plans that match your defined filters in the mabl cloud
- mabl tests run-cloud - runs one or more tests in the mabl cloud
Local and self-hosted CI environments support sequential execution only. To run tests in local and CI environments, use these commands:
- mabl tests run - runs one or more browser or API tests on your local machine or CI environment
- mabl tests run-mobile - runs one or more mobile tests on your local machine
Read on to learn more about how to trigger tests from the mabl CLI.
mabl deployments create
Use the mabl deployments create command to trigger a parallel execution in the mabl cloud of all plans that match a specified set of filters. Filters can include application, environment, and plan labels.
Because deployment events can execute many tests in parallel, this command is ideal for getting robust validation on a large set of tests.
Options
At a minimum, use one of the following options to specify which plans to run:
| Option | Description |
|---|---|
--application-id and/or --environment-id
|
Use at least one of these options to specify which plans you want to run. |
--plan-labels |
Optional: run plans that match any label in the space-delimited list of plan labels. |
The mabl deployments create command supports a large number of options for configuring overrides, branches, browser settings, output, and more. For an exhaustive list of all options, please refer to the documentation in the mabl CLI by running mabl deployments create -h.
Refer to the article on deployment events in the mabl CLI for more details on options and sample commands.
Output
By default, the CLI outputs a link to view the results of the deployment event in the mabl app.
Examples
# Trigger a deployment of all plans with a "regression" label
mabl deployments create -e {environment-id} --labels regression
# Use a URL override to run plans against a preview environment
mabl deployments create -e {environment-id} --app-url {url-override}mabl tests run-cloud
Use the mabl tests run-cloud command to trigger one or more tests to run in parallel in the mabl cloud.
Because cloud runs collect more detailed diagnostics than local or CI runs, this command is ideal for investigating test failures in more detail.
Options
At a minimum, include one of the following options to specify which test(s) to run:
| Option | Description |
|---|---|
--id |
Run the test that matches this ID |
--labels |
Run any tests that match the space-delimited list of labels |
--exclude-labels |
Exclude any tests that match the space-delimited list of labels. Use in conjunction with the --labels option. |
To configure additional settings or overrides, include more options in the command. For an exhaustive list of all options, please refer to the documentation in the mabl CLI by running mabl tests run-cloud -h.
Commonly used options include the following:
| Option | Description |
|---|---|
--url, -u, --app-url, or --api-url
|
Run the test against a specific URL |
--environment-id, -e
|
Run the test using the variables and latest find information from a specific environment |
--credentials-id, --creds
|
Associate the test with mabl credentials |
--basic-auth-credentials-id, --basic-creds
|
Associate the test with Basic Auth credentials |
--browsers, -b
|
Specify which browser to run the test against. The default is Chrome. |
--mabl-branch |
Run a branched version of the test. |
Output
The mabl CLI outputs a link to view test output in the app.
Examples
# Trigger a cloud run of a test that failed locally
mabl tests run-cloud --id {test-id}
# Trigger cloud runs of tests that failed in a CI build
mabl tests run-cloud --labels follow-upmabl tests run
Use the mabl tests run command to trigger one or more browser or API tests to run on your local machine or in a CI environment. The mabl CLI returns quick pass/fail feedback without consuming your cloud credits.
Because local and CI runs give quick feedback, this command is ideal for rapid iteration and debugging on individual tests during the development phase of your pipeline. Sequential execution makes this command less efficient for larger test suites.
Options
At a minimum, include one of the follow options to specify which test(s) to run:
| Option | Description |
|---|---|
--id |
Run the test that matches this ID |
--run-id |
ID of the test run to pull configuration from |
--labels |
Run any tests that match the space-delimited list of labels |
--exclude-labels |
Exclude any tests that match the space-delimited list of labels. Use in conjunction with the --labels option. |
--from-plan-id |
Run tests associated with a specific plan. Note that this is not the same as a plan run in the cloud. Advanced plan configurations, such as stages, browser settings, shared variables, and DataTable/credential overrides, are not applied. Tests run sequentially and use the same base URL. |
Create designated plans to use with --from-plan-id
The --from-plan-id option does not apply advanced plan configurations or parallel execution. If you use this option, we recommend creating designated plans for local runs that only contain a single stage. The tests in designated local plans should be of only one type - browser or API - since all tests use the same base URL.
To configure additional settings or overrides, include more options in the command. For an exhaustive list of all options, please refer to the documentation in the mabl CLI by running mabl tests run -h.
Commonly used options include the following:
| Option | Description |
|---|---|
--url |
Run the test against a specific URL |
--environment-id |
Run the test using the variables and latest find information from a specific environment. |
--headless |
Run the test without visually rendering the browser. |
--credentials-id, --creds
|
Associate the test with mabl credentials. |
--data-table-id or --scenario-id
|
Run the test with one or all scenarios of a DataTable |
--basic-auth-credentials-id, --basic-creds
|
Associate the test with Basic Auth credentials |
--mabl-branch |
Run a branched version of the test. |
Browser support
Browser test runs use the Chrome browser installed on your machine.
To use a different chromium browser, update the browser path using the command mabl config set browser.path {path-to-browser}. Keep in mind that incorrectly configuring the browser path may break local test execution.
Output
By default, the mabl CLI only outputs results directly in the terminal. To publish results of test output to the mabl app, include the --reporter option with the argument mabl:
mabl tests run --id {test-id} --environment-id {env-id} --application-id {app-id} --reporter mablFor more information on publishing local run results to mabl, check out the article on the Unified Reporter.
Examples
# Trigger a headless local run
mabl tests run --id {test-id} --headless true
# Trigger a local run for tests that match labels
mabl tests run --labels smoke needs-review
# Trigger local test runs associated with a plan
mabl tests run --from-plan-id {plan-id}mabl tests run-mobile
Use mabl tests run-mobile to run one or more mobile tests on your local machine.
This command is ideal for getting quick pass/fail feedback on mobile-specific functionality during development. Sequential execution makes this command less efficient for larger test suites. Local mobile runs do not consume cloud credits.
Setup
Executing mobile tests locally requires access to simulators or emulators from a mobile development IDE:
- iOS - to run iOS tests locally, your machine must have XCode installed with available simulators.
- Android - to run Android tests locally, your machine must have Android Studio installed with available emulators.
Before triggering test runs, run the command mabl config install mobile-tools to download and install the necessary software components for interacting with your Android and iOS apps.
Options
The mabl tests run-mobile command requires two separate pieces of information to execute a test:
- The mobile test(s) to be executed
- The mobile build file to run the test against
To specify which mobile test(s) to run, use one of the following options:
| Option | Description |
|---|---|
--id |
Run the mobile test that matches this ID |
--run-id |
ID of the test run to pull configuration from. If you use this option, you don't need to specify a mobile build file since it's already included in the run configuration. |
--labels |
Run any mobile tests that match the space-delimited list of labels |
--exclude-labels |
Exclude any tests that match the space-delimited list of labels. Use in conjunction with the --labels option. |
--from-plan-id |
Run tests associated with a specific plan. Note that this is not the same as a plan run in the cloud: advanced plan configurations, such as stages, DataTables, or shared variables, are not applied. Tests run sequentially and use the same mobile build file. We recommend that you only use this option with mobile-only plans that can run in a single stage. |
To specify the mobile build file, use one of the following options:
| Option | Description |
|---|---|
--build-file-id |
ID of a build file from your mabl workspace. |
--build-file-path, --app-file
|
Path to a local mobile application binary. Use in conjunction with the --platform option |
--platform |
Platform associated with the build: "ios" or "android". Required when using a local build file path. Not necessary when using --build-file-id. |
To run the test on a specific device, include the --device option:
-
iOS - to get a list of IDs for available devices, run
xcrun simctl list devicesin your terminal. For example, to run a mobile test locally on an iPhone 15 Pro, the option would be--device 7F5FF587-4F9F-441D-BC5B-681E0AF2EA4E. -
Android - to get a list of available devices, such as
Pixel_6_API_33, run./emulator -list-avdsfrom the directory where theemulatorexecutable is installed. This location depends on your OS. Some possible locations include:- macOS:
$HOME/Library/Android/sdk/emulator - Linux:
$HOME/Android/sdk/emulatoror$HOME/Android/Sdk/emulator - Windows:
$USERPROFILE/AppData/Local/Android/Sdk
- macOS:
For a complete list of all options, please refer to the documentation in the mabl CLI by running mabl tests run-mobile -h.
Output
The mabl CLI outputs test output directly in the terminal.
Examples
# Run a mobile test using configuration from a cloud run
mabl tests run-mobile --run-id {test-run-id}
# Run a set of iOS tests associated with the label "wip"
mabl tests run-mobile --labels wip --build-file-path {relative-path-to-file} --platform ios
# Run an Android test on a Pixel 9 device
mabl tests run-mobile --id {test-id} --build-file-id {build-file-id} --device Pixel_9_Pro_API_36