Using the mabl CLI

Getting started

To interact with most functions, you need to authenticate first:

mabl auth login

To check if you are successfully authenticated and find out when your session expires, run:

$ mabl auth info
> Logged in as user [[email protected]]
> Login expires in [3 hours, 23 minutes]

Here are the supported top-level commands that should give you an idea of the underlying pattern for working with the CLI:

$ mabl <command>

Commands:
  mabl applications <command>  Manage your applications
  mabl auth <command>          Manage auth for the mabl CLI
  mabl branches <command>      Manage mabl branches
  mabl config <command>        Configure defaults for the mabl CLI
  mabl credentials <command>   Manage your testing credentials
  mabl deployments <command>   Manage mabl deployments
  mabl environments <command>  Manage your environments
  mabl tests <command>         Manage tests
  mabl test-runs <command>     Manage existing test runs
  mabl workspaces <command>    Manage your workspaces

Options:
  --version   Show version number  [boolean]
  -h, --help  Show help  [boolean]

Read full docs @ https://help.mabl.com/docs/mabl-cli

To see help on all supported commands or available for options for a given command, use the --help flag as for example:

mabl --help
mabl <command> --help

Querying your workspace

You can use mabl CLI to get information about different entities - including workspaces, applications, environments, and tests - and their IDs. To get a list of entities of each type run one of the following:

mabl workspaces list 
mabl branches list 
mabl applications list
mabl environments list
mabl deployments list
mabl credentials list
mabl tests list

To get more detailed information about a specific entity, such as an environment, run one of the following:

mabl workspaces describe <workspace-id>
mabl applications describe <application-id>
mabl environments describe <environment-id>

You can then use this information with other CLI commands, as for example to create a mabl deployment event that triggers regression test plan runs.

Creating tests

Using the mabl CLI, you can create new tests quickly directly through the command line of your Terminal without having to go through the mabl web app. For example, if you are a software developer working on a new feature, you can quickly create a new end-to-end test against your local environment directly from the terminal within Visual Studio Code.

To a create a test from your command line, run the following command by providing a starting URL for the test and a name as follows. Note that in order to use spaces in the test name, you need to wrap it up in quotation marks (e.g. "My new test").

$ mabl tests create <url> <test name>

>Creating test:  My new mabl test
>Url:  https://example.com
>Workspace:  1111aa_222bbb333ccc44-w

Running this command will open up an instance of your local Chrome browser with the desktop app Trainer. The browser will navigate to the specified URL and you can use the Trainer to create your test steps. When done creating your test, simply save your work and close the Trainer which will return you back to the command line.

To learn more about all the different options for creating a test, add the --help option to the tests create command as show below:

$ mabl tests create --help

Create a test using the mabl Trainer

Positionals:
  url        The url to test  [string] [required]
  test-name  The name of the test  [string] [required]

Options:
  --version                   Show version number  [boolean]
  -h, --help                  Show help  [boolean]
  --desc, --test-description  Description for the test  [string] [default: ""]
  --width                     Set the browser width in pixels  [number] [default: 1366]
  --height                    Set the browser height in pixels  [number] [default: 768]
  --mabl-branch               Branch to run the mabl test against  [string]
  --auto-branch               Create a mabl branch when a mabl-branch target is specified that does not exist  [boolean] [default: false]
  -w, --workspace-id          Workspace to create test in  [string]
  --creds, --credentials      Credentials ID to run the test with  [string]

You can see all created tests in the workspace by running the mabl tests list command.

Editing tests

To edit a test locally you'll need to provide either a test id or a test run id. These can be found on the test details page under the CLI info button or on the test output page under the actions menu in the upper right. Once you have the needed ID, use the following command to launch a test editing session.

mabl tests edit --id <id>

# Alternatively supply a test-run-id
mabl tests edit --run-id <id>

A fresh Chrome window will then open with a mabl trainer editing session initialized inside of it. Other config options are available and can be seen by running the command with the -h flag for help

Running tests

You can use the mabl CLI to run tests in three different ways:

1. On your local machine for quick test validation but with limited diagnostics for troubleshooting failures
2. In the mabl cloud for full diagnostics and parallelism
3. As part of your CI/CD pipeline on a deployment event to get feedback on the build

Run tests locally

To run a test locally, you will first need to obtain the ID of the test you want to run. From your command line, you can list all tests in the workspace with mabl tests list and copy the desired test ID. Alternatively, you can find the ID in the Test Details page.

Once you have the test ID, you can trigger a local run via the CLI with the following command, which will run the test in Chrome on your local machine in full browser mode, not headless.

mabl tests run --id <id>

📘

Overriding the default app.url

Local tests triggered from the mabl CLI run against the test’s original URL. To run the test on a different url (e.g. localhost), you can add the optional argument --url and specify the URL: mabl tests run --id <id> --url https://www.example.com

Alternatively you can use the test run ID to run a test locally using the same configuration as a previous cloud run.

mabl tests run --run-id <id>

You can monitor the test run progress in your terminal while watching the test interacting with the app under test in your browser.

For even faster execution, you can run the tests locally in headless mode using the --headless option.

mabl tests run --id <test id> --headless

Run tests in the cloud

If you've used the CLI to run a test locally and now want to run in the cloud to capture rich data for every step (e.g. screenshots, DOM snapshots, network activity, etc.). You can trigger a test run in the mabl cloud with the following command:

mabl tests run-cloud --id <id> --url <url>

You can grab the test ID as described for the local test runs and specify the URL that the test needs to start from. You can also provide a deployment ID instead of the option to run the test with the same URL and browser(s) as a given deployment test run. You can read more about running tests on deployments below.

Selecting tests by label and or branch

You can use labels to include and exclude tests from consideration. For example, you can run all tests with test label feature-x and feature-y, but exclude tests with a work in process label (e.g. wip) with the following expression:

# Launch all tests labeled 'feature-x' OR 'feature-y'
mabl tests run-cloud --labels feature-x feature-y

# Launch all tests with labels, but exclude WIP labeled tests
mabl tests run-cloud --labels feature-x feature-y --exclude-labels wip

You can also use branches to select which test variants to run using --mabl-branch <branch-name>. Additionally, you can run only tests that were modified on a branch (rather than the rest of the branch) with the --branch-changes-only flag, as shown below:

# Constrain tests to a specific branch with a specific test id
mabl tests run-cloud --mabl-branch my-feature-branch --id <id>

# Constrain tests to only a specific branch AND only tests edited on that branch that match a label called WIP
mabl tests run-cloud --mabl-branch my-feature-branch --branch-changes-only --labels WIP

To see all options for the run-cloud command, just use the --help option:

$ mabl tests run-cloud --help  

Run test(s) in the mabl cloud

Options:
  --version              Show version number  [boolean]
  -h, --help             Show help  [boolean]
  --id                   the id of the test to run  [string]
  --deployment-id, -d    Deployment to run the mabl tests against  [string]
  --url, -u              URL to run the mabl test against  [string]
  --browsers, -b         Space delimited browsers to test against (e.g. "chrome firefox")  [array] [required]
  --api-key, -k          API key (found in the mabl web app)  [string]
  --mabl-branch          Mabl branch to run test against  [string]
  --branch-changes-only  Only execute tests changed on specified branch  [boolean] [default: false]
  --revision, --rev      Code revision hash (application under test)  [string]
  --labels               Space delimited test labels. Run tests that match any label.  [array]
  --exclude-labels       Space delimited test labels. Exclude tests that match any label.  [array]
  --workspace-id, -w     Workspace to run against  [string]
  --prompt               Prompt to confirm execution selections  [boolean] [default: true]
  --no-prompt, --yes     Do not prompt to confirm execution selections  [boolean]

Examples:
  mabl tests run-cloud --id <id>                   run test by id
  mabl tests run-cloud --labels <label1> <label2>  run tests by test label

Running tests on deployment

To trigger a deployment event that runs tests for a given application and environment, run the following command:

mabl deployments create --application-id <application_id> --environment-id <environment_id>

# Specify source control information to use with mabl GitHub App
mabl deployments create \
 --application-id <application_id> \
 --environment-id <environment_id> \
 --revision 41d502b463a37e87047148a6000411a1e1bf9a79 \
 --repository-url [email protected]:example-org/example-repo.git

This will trigger a run of all mabl plans for that app environment that have been configured to run on deployment events and are not disabled. You can further narrow down which plans are run on deployment by using the --labels option.

Note that passing the --revision and --repository-url flags allow coordination between tests started with the mabl CLI and the mabl GitHub integration. If you provide these values, and have the GitHub App installed, then GitHub Checks will appear with detailed tests results in your GitHub commits and pull requests. Read more about the mabl GitHub integration setup.

To get the IDs for your mabl application and environment, go to Settings --> APIs in the mabl app, choose Deployment Events API, select Application and Environment and scroll to the bottom of the page where you will see a generated CURL command showing the respective IDs.

To trigger the tests to run against an ephemeral or preview environment, you can use the --app-url and --api-url arguments to specify an alternate application and API URL, respectively. These arguments will update the app.url and api.url variables in your tests.

You can see all deployment events in the mabl app by going to Results --> By Deployment.

To see all deployment events in mabl, run:

mabl deployments list

And to get details for a specific deployment, run:

mabl deployments describe <deployment_id>

For all supported deployment commands and options, run:

mabl deployments --help

Export test data

You can use the mabl CLI to export all outputs for a given test run. Customers typically use the screenshots to have product design discussions with their team or for compliance reasons. For example, if you have a test that tests the user signup and onboarding flow of your app, you can get the screenshots and discuss with your team ways to optimize the user experience to increase signups.

Supported test output types are:

• Chrome Traces
• Console Logs
• DOM snapshots
• HAR logs (network requests)
• Screenshots

To export, first you'll need the test run id. Please see the Obtaining mabl Resource IDs for the CLI section below for how to get a test run id from the test output page.

Next, go to your terminal and run the following mabl CLI command using the test run id that you copied previously:

# Export screenshots only
mabl test-runs export <test-run-id> --types screenshots

# Export all output types
mabl test-runs export <test-run-id> --types hars traces doms console_logs screenshots

This will package all screenshots as a ZIP file and download it to your computer in the working directory from which you run the CLI command. Note that if there are many steps in the test, downloading the screenshots may take minutes and result in files of 100MiB or larger.

To see all export options, run:

mabl test-runs export --help

Obtaining mabl Resource IDs for the CLI

The mabl web app offers an easy way to get the needed test IDs, test run IDs, and more for use in the mabl CLI through the mabl CLI resource IDs modal

Test details page

You can easily get the test ID from the test details page using the mabl CLI resource ID viewer. Click the terminal icon in the upper left toolbar on the page to open the modal. The modal will contain the test-id along with other relevant IDs to be used in the CLI. Use the copy button to easily copy the IDs.

Test output page

The test output page also contains the CLI resource ID viewer. You can access the same modal from the actions menu in the upper right corner next to the "Edit steps" button. From the dropdown menu select "CLI info" to open the modal which will now contain the run-id for that test output.