mabl CLI overview

mabl CLI is a command-line tool that helps you integrate mabl with your CI/CD pipeline and developer workflow. It is distributed as a Node.js package available on npmjs.com.

Use cases

  • Create, edit, and run tests from your command line
  • Integrate testing as part of your CI/CD build process
  • Run smoke and regression tests on deployment
  • Import from Selenium tests into mabl
  • Query your workspaces, applications, environments and tests
  • Work with mabl test branches
  • See deployment test results
  • Download screenshots for a given test
  • Export from tests to Selenium IDE format

If you would like to integrate the mabl CLI into your CI/CD pipeline, please refer to our documentation for the mabl Runner and JavaScript testing frameworks.

Should you need support for additional use cases, let us know by posting on our feedback portal.

Beta release

mabl CLI is currently in beta and under rapid development. APIs are subject to change without notice prior to the general availability release.

Getting started

To get started, you will need to to install Node.js version 10.0.0 or higher. If you are using a Mac, we recommend installing node via the Homebrew package manager. With node installed, install the mabl CLI by running the following command your terminal:

npm i -g @mablhq/mabl-cli

To see the installed version of the CLI run:

mabl --version

Once you install the CLI, you will need to authenticate using either your mabl login credentials or workspace API key. If you are using the CLI on your local machine, you can simply login using your mabl credentials:

mabl auth login

On the other hand, if you want to use the mabl CLI as part of your build process, you should authenticate with the API key. To get the API key, go to the mabl app -> Settings -> APIs to copy the key and use with the following command:

mabl auth activate-key <YOUR-API-KEY>

Running tests on your local machine

To run tests locally, including using headless mode, please authenticate with your mabl login credentials since API authentication is only meant for running tests on deployment as part of your CI/CD pipeline.

To check if you are successfully authenticated, run:

$ mabl auth info
> Logged in as user [[email protected]]
> Login expires in [3 hours, 23 minutes]

Here are the supported top-level commands that should give you an idea of the underlying pattern for working with the CLI:

$ mabl <command>

Commands:
  mabl applications <command>  Manage your applications
  mabl auth <command>          Manage auth for the mabl CLI
  mabl branches <command>      Manage mabl branches
  mabl config <command>        Configure defaults for the mabl CLI
  mabl credentials <command>   Manage your testing credentials
  mabl deployments <command>   Manage mabl deployments
  mabl environments <command>  Manage your environments
  mabl tests <command>         Manage tests
  mabl test-runs <command>     Manage existing test runs
  mabl workspaces <command>    Manage your workspaces

Options:
  --version   Show version number  [boolean]
  -h, --help  Show help  [boolean]

Read full docs @ https://help.mabl.com/docs/mabl-cli

To see help on all supported commands or available for options for a given command, use the --help flag as for example:

mabl --help
mabl <command> --help

Querying your workspace

You can use mabl CLI to get information about your workspaces, applications, environments, tests, etc. To get a list of entities of each type run one of the following:

mabl workspaces list 
mabl branches list 
mabl applications list
mabl environments list
mabl deployments list
mabl credentials list
mabl tests list

To get more detailed information about a specific entity, such as an environment, run one of the following:

mabl workspaces describe <workspace-id>
mabl applications describe <application-id>
mabl environments describe <environment-id>

You can then use this information with other CLI commands, as for example to create a mabl deployment event that triggers regression test plan runs.

Creating tests

Using the mabl CLI you can create new tests quickly directly through the command line of your Terminal without having to go through the mabl web app. For example, if you are a software developer working on a new feature, you can quickly create a new end-to-end test against your local environment directly from the terminal within Visual Studio Code.

To a create a test from your command line, run the following command by providing a starting URL for the test and a name as follows. Note that in order to use spaces in the test name, you need to wrap it up in quotation marks (e.g. "My new test").

$ mabl tests create <url> <test name>

>Creating test:  My new mabl test
>Url:  https://example.com
>Workspace:  1111aa_222bbb333ccc44-w

Running this command will open up an instance of your local Chrome browser with the mabl Trainer extension installed. The browser will navigate to the specified URL and you can use the Trainer to create your test steps. When done creating your test, simply save your work and close the Trainer which will return you back to the command line.

To learn more about all the different options for creating a test, add the --help option to the tests create command as show below:

$ mabl tests create --help

Create a test using the mabl Trainer

Positionals:
  url        The url to test  [string] [required]
  test-name  The name of the test  [string] [required]

Options:
  --version                   Show version number  [boolean]
  -h, --help                  Show help  [boolean]
  --desc, --test-description  Description for the test  [string] [default: ""]
  --width                     Set the browser width in pixels  [number] [default: 1366]
  --height                    Set the browser height in pixels  [number] [default: 768]
  --mabl-branch               Branch to run the mabl test against  [string]
  --auto-branch               Create a mabl branch when a mabl-branch target is specified that does not exist  [boolean] [default: false]
  -w, --workspace-id          Workspace to create test in  [string]
  --creds, --credentials      Credentials ID to run the test with  [string]

You can see all created tests in the workspace by running the mabl tests list command.

Editing tests

To edit a test locally you'll need to provide either a test id or a test run id. These can be found on the test details page under the CLI info button or on the test output page under the actions menu in the upper right (see Running tests section for more info on menu location). Once you have the needed ID use the following command to launch a test editing session.

mabl tests edit --id <id>

# Alternatively supply a test-run-id
mabl tests edit --run-id <id>

A fresh Chrome window will then open with a mabl trainer editing session initialized inside of it. Other config options are available and can be seen by running the command with the -h flag for help

Running tests

You can use the mabl CLI to run tests in three different ways:
1) On your local machine for quick test validation but with limited diagnostics for troubleshooting failures
2) In the mabl cloud for full diagnostics and parallelism
3) As part of your CI/CD pipeline on a deployment event to get feedback on the build

Run tests locally

To run a test locally, you will first need to obtain the ID of the test you want to run. From your command line, you can list all tests in the workspace with mabl tests list and copy the desired test ID. Alternatively, you can find the IDs in the mabl web app. See the Obtaining mabl Resource IDs for the CLI section below for more info.

Once you have the test ID, you can trigger a local run via the CLI with the following command, which will run the test in Chrome on your local machine in full browser mode, not headless.

mabl tests run --id <id>

Alternatively you can use the test run ID to run a test locally using the same configuration as a previous cloud run. See the Obtaining mabl Resource IDs for the CLI section below for more info on how to get the test run ID.

mabl tests run --run-id <id>

You can monitor the test run progress in your terminal while watching the test interacting with the app under test in your browser.

Run tests in the cloud

Let's say you've used the CLI to run a test locally and want to now run in the cloud to capture the rich data for every step (e.g. screenshots, DOM snapshots, network activity, etc.). You can trigger a test run in the mabl cloud with the following command:

mabl tests run-cloud --id <id> --url <url>

You can grab the test ID as described for the local test runs and specify the URL that the test needs to start from. You can also provide a deployment ID instead of the <url> option to run the test with thee same URL and browser(s) as a given deployment test run. You can read more about running tests on deployments below.

Selecting tests by label and or branch

You can use labels to include and exclude tests from consideration. For example, you can run all tests with test label feature-x and feature-y, but exclude tests with a work in process label (e.g. wip) with the following expression:

# Launch all tests labeled 'feature-x' OR 'feature-y'
mabl tests run-cloud --labels feature-x feature-y

# Launch all tests with labels, but exclude WIP labeled tests
mabl tests run-cloud --labels feature-x feature-y --exclude-labels wip

You can also use branches to select which test variants to run using --mabl-branch <branch-name>. Additionally, you can run only tests that were modified on a branch (rather than the rest of the branch) with the --branch-changes-only flag, as shown below:

# Constrain tests to a specific branch with a specific test id
mabl tests run-cloud --mabl-branch my-feature-branch --id <id>

# Constrain tests to only a specific branch AND only tests edited on that branch that match a label called WIP
mabl tests run-cloud --mabl-branch my-feature-branch --branch-changes-only --labels WIP

To see all options for the run-cloud command, just use the --help option:

$ mabl tests run-cloud --help  

Run test(s) in the mabl cloud

Options:
  --version              Show version number  [boolean]
  -h, --help             Show help  [boolean]
  --id                   the id of the test to run  [string]
  --deployment-id, -d    Deployment to run the mabl tests against  [string]
  --url, -u              URL to run the mabl test against  [string]
  --browsers, -b         Space delimited browsers to test against (e.g. "chrome firefox")  [array] [required]
  --api-key, -k          API key (found in the mabl web app)  [string]
  --mabl-branch          Mabl branch to run test against  [string]
  --branch-changes-only  Only execute tests changed on specified branch  [boolean] [default: false]
  --revision, --rev      Code revision hash (application under test)  [string]
  --labels               Space delimited test labels. Run tests that match any label.  [array]
  --exclude-labels       Space delimited test labels. Exclude tests that match any label.  [array]
  --workspace-id, -w     Workspace to run against  [string]
  --prompt               Prompt to confirm execution selections  [boolean] [default: true]
  --no-prompt, --yes     Do not prompt to confirm execution selections  [boolean]

Examples:
  mabl-dev tests run-cloud --id <id>                   run test by id
  mabl-dev tests run-cloud --labels <label1> <label2>  run tests by test label

Running tests on deployment

To trigger a deployment event that runs tests for a given application and environment, run the following command:

mabl deployments create --application-id <application_id> --environment-id <environment_id>

# Specify source control information to use with mabl GitHub App
mabl deployments create \
 --application-id <application_id> \
 --environment-id <environment_id> \
 --revision 41d502b463a37e87047148a6000411a1e1bf9a79 \
 --repository-url git@github.com:foo-org/bar-repo.git

This will trigger a run of all mabl plans for that app environment that have been configured to run on deployment events and are not disabled. You can further narrow down which plans are run on deployment by using the --labels option.

Note that passing the --revision and --repository-url flags allow coordination between tests started with the mabl CLI and the mabl GitHub App. If you provide these values, and have the GitHub App installed, then GitHub Checks will appear with detailed tests results in your GitHub commits and pull requests. Read more about the mabl GitHub integration setup.

To get the IDs for your mabl application and environment, go to Settings --> APIs in the mabl app, choose Deployment Events API, select Application and Environment and scroll to the bottom of the page where you will see a generated CURL command showing the respective IDs.

You can see all deployment events in the mabl app by going to Results --> By Deployment.

To see all deployment events in mabl, run:

mabl deployments list

And to get details for a specific deployment, run:

mabl deployments describe <deployment_id>

For all supported deployment commands and options, run:

mabl deployments --help

Export test data

You can use the mabl CLI to export all screenshots for a given test run. Customers typically use the screenshots to have product design discussions with their team or for compliance reasons. For example, if you have a test that tests the user signup and onboarding flow of your app, you can get the screenshots and discuss with your team ways to optimize the user experience to increase signups.

First you'll need the test run id. Please see the Obtaining mabl Resource IDs for the CLI section below for how to get a test run id from the test output page.

Next, go to your terminal and run the following mabl CLI command using the test run id that you copied previously:

mabl test-runs export <test-run-id>

This will package all screenshots as a ZIP file and download it to your computer in the working directory from which you run the CLI command. Note that if there are many steps in the test, downloading the screenshots may take minutes and result in files of 100MiB or larger.

To see all export options, run:

mabl test-runs export --help

Updating the CLI tool

To update your version of the mabl CLI tool, run:

npm install -g @mablhq/[email protected]

Have another use case in mind for the mabl CLI? Please let us know by posting on our feedback portal. We love acting on user feedback.

Configuration

You can get, set, and list available configuration options using the list subcommand. This command can be used to set the current workspace default in the mabl CLI amongst other options.

# view current configuration values 
mabl config list

# set a configuration value
mabl config set workspace <WORKSPACE_ID>

# get a configuration value
mabl config get workspace

Proxy Configuration

The mabl CLI supports forward proxies. To use a proxy, set the http.proxy configuration value to the URL for your proxy server. The URL can contain basic authentication credentials and port information.

If you receive an error that the URL you are specifying is invalid, please confirm that you are using Node version 10 or greater.

# add a proxy with no credentials on port 3128 
mabl config set http.proxy http://proxy.mycompany.com:3128
 
# add a proxy using credentials on port 3128 
mabl config set http.proxy http://"username:password"@proxy.mycompany.com:3128
 
# delete the current proxy setting 
mabl config delete http.proxy

If you receive a 407 error when logging in, this is a proxy authentication error. Ensure your password and username have been set correctly above.

Proxy settings only apply to mabl traffic

Note that if the above configuration option is set, only mabl-specific requests will go through the forwarding proxy. For local test executions, all traffic to the application under test will use the proxy settings of the browser or the operating system.

SSL Verification

By default, the CLI verifies server certificates are issued by a known certificate authority. Certificate verification may fail if your proxy is intercepting HTTPS traffic. If you are using a proxy and SSL verification is failing, you can disable certificate verification as shown below. We suggest you do this only if required.

# disable SSL verification 
mabl config set http.sslVerify false
 
# delete the current ssl verification setting to use the default 
mabl config delete http.sslVerify

Obtaining mabl Resource IDs for the CLI

The mabl web app offers an easy way to get the needed test IDs, test run IDs, and more for use in the mabl CLI through the mabl CLI resource IDs modal

the mabl CLI resource IDs modal

the mabl CLI resource IDs modal

Test details page

You can easily get the test ID from the test details page using the mabl CLI resource ID viewer. Click the terminal icon in the upper left toolbar on the page to open the modal. The modal will contain the test-id along with other relevant IDs to be used in the CLI. Use the copy button to easily copy the IDs.

CLI info button

CLI info button

Test output page

The test output page also contains the CLI resource ID viewer. You can access the same modal from the actions menu in the upper right corner next to the "Edit steps" button. From the dropdown menu select "CLI info" to open the modal which will now contain the run-id for that test output.

The actions menu location on the test output page

The actions menu location on the test output page

Updated 8 days ago


Related resources

Selenium IDE export

mabl CLI overview


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.