mabl CLI is a command-line tool that helps you integrate mabl with your CI/CD pipeline and developer workflow. It is distributed as a Node.js package available on npmjs.com.
- Create, edit, and run tests from your command line
- Integrate testing as part of your CI/CD build process
- Run smoke and regression tests on deployment
- Import from Selenium tests into mabl
- Query your workspaces, applications, environments and tests
- Work with mabl test branches
- See deployment test results
- Download screenshots for a given test
- Export from tests to Selenium IDE format
Should you need support for additional use cases, let us know by posting on our feedback portal.
mabl CLI is currently in beta and under rapid development. APIs are subject to change without notice prior to the general availability release.
To get started, you will need to to install Node.js version 10.0.0 or higher. If you are using a Mac, we recommend installing node via the Homebrew package manager. With node installed, install the mabl CLI by running the following command your terminal:
npm i -g @mablhq/mabl-cli
To see the installed version of the CLI run:
Once you install the CLI, you will need to authenticate using either your mabl login credentials or workspace API key. If you are using the CLI on your local machine, you can simply login using your mabl credentials:
mabl auth login
On the other hand, if you want to use the mabl CLI as part of your build process, you should authenticate with the API key. To get the API key, go to the mabl app -> Settings -> APIs to copy the key and use with the following command:
mabl auth activate-key <YOUR-API-KEY>
Running tests on your local machine
To run tests locally, including using headless mode, please authenticate with your mabl login credentials since API authentication is only meant for running tests on deployment as part of your CI/CD pipeline.
To check if you are successfully authenticated, run:
$ mabl auth info > Logged in as user [[email protected]] > Login expires in [3 hours, 23 minutes]
Here are the supported top-level commands that should give you an idea of the underlying pattern for working with the CLI:
$ mabl <command> Commands: mabl applications <command> Manage your applications mabl auth <command> Manage auth for the mabl CLI mabl branches <command> Manage mabl branches mabl config <command> Configure defaults for the mabl CLI mabl credentials <command> Manage your testing credentials mabl deployments <command> Manage mabl deployments mabl environments <command> Manage your environments mabl tests <command> Manage tests mabl test-runs <command> Manage existing test runs mabl workspaces <command> Manage your workspaces Options: --version Show version number [boolean] -h, --help Show help [boolean] Read full docs @ https://help.mabl.com/docs/mabl-cli
To see help on all supported commands or available for options for a given command, use the
--help flag as for example:
mabl --help mabl <command> --help
You can use mabl CLI to get information about your workspaces, applications, environments, tests, etc. To get a list of entities of each type run one of the following:
mabl workspaces list mabl branches list mabl applications list mabl environments list mabl deployments list mabl credentials list mabl tests list
To get more detailed information about a specific entity, such as an environment, run one of the following:
mabl workspaces describe <workspace-id> mabl applications describe <application-id> mabl environments describe <environment-id>
You can then use this information with other CLI commands, as for example to create a mabl deployment event that triggers regression test plan runs.
Using the mabl CLI you can create new tests quickly directly through the command line of your Terminal without having to go through the mabl web app. For example, if you are a software developer working on a new feature, you can quickly create a new end-to-end test against your local environment directly from the terminal within Visual Studio Code.
To a create a test from your command line, run the following command by providing a starting URL for the test and a name as follows. Note that in order to use spaces in the test name, you need to wrap it up in quotation marks (e.g. "My new test").
$ mabl tests create <url> <test name> >Creating test: My new mabl test >Url: https://example.com >Workspace: 1111aa_222bbb333ccc44-w
Running this command will open up an instance of your local Chrome browser with the mabl Trainer extension installed. The browser will navigate to the specified URL and you can use the Trainer to create your test steps. When done creating your test, simply save your work and close the Trainer which will return you back to the command line.
To learn more about all the different options for creating a test, add the
--help option to the
tests create command as show below:
$ mabl tests create --help Create a test using the mabl Trainer Positionals: url The url to test [string] [required] test-name The name of the test [string] [required] Options: --version Show version number [boolean] -h, --help Show help [boolean] --desc, --test-description Description for the test [string] [default: ""] --width Set the browser width in pixels [number] [default: 1366] --height Set the browser height in pixels [number] [default: 768] --mabl-branch Branch to run the mabl test against [string] --auto-branch Create a mabl branch when a mabl-branch target is specified that does not exist [boolean] [default: false] -w, --workspace-id Workspace to create test in [string] --creds, --credentials Credentials ID to run the test with [string]
You can see all created tests in the workspace by running the
mabl tests list command.
To edit a test locally you'll need to provide either a test id or a test run id. These can be found on the test details page under the CLI info button or on the test output page under the actions menu in the upper right (see Running tests section for more info on menu location). Once you have the needed ID use the following command to launch a test editing session.
mabl tests edit --id <id> # Alternatively supply a test-run-id mabl tests edit --run-id <id>
A fresh Chrome window will then open with a mabl trainer editing session initialized inside of it. Other config options are available and can be seen by running the command with the
-h flag for help
You can use the mabl CLI to run tests in three different ways:
1) On your local machine for quick test validation but with limited diagnostics for troubleshooting failures
2) In the mabl cloud for full diagnostics and parallelism
3) As part of your CI/CD pipeline on a deployment event to get feedback on the build
To run a test locally, you will first need to obtain the ID of the test you want to run. From your command line, you can list all tests in the workspace with
mabl tests list and copy the desired test ID. Alternatively, you can find the IDs in the mabl web app. See the Obtaining mabl Resource IDs for the CLI section below for more info.
Once you have the test ID, you can trigger a local run via the CLI with the following command, which will run the test in Chrome on your local machine in full browser mode, not headless.
mabl tests run --id <id>
Alternatively you can use the test run ID to run a test locally using the same configuration as a previous cloud run. See the Obtaining mabl Resource IDs for the CLI section below for more info on how to get the test run ID.
mabl tests run --run-id <id>
You can monitor the test run progress in your terminal while watching the test interacting with the app under test in your browser.
Let's say you've used the CLI to run a test locally and want to now run in the cloud to capture the rich data for every step (e.g. screenshots, DOM snapshots, network activity, etc.). You can trigger a test run in the mabl cloud with the following command:
mabl tests run-cloud --id <id> --url <url>
You can grab the test ID as described for the local test runs and specify the URL that the test needs to start from. You can also provide a deployment ID instead of the <url> option to run the test with thee same URL and browser(s) as a given deployment test run. You can read more about running tests on deployments below.
You can use labels to include and exclude tests from consideration. For example, you can run all tests with test label
feature-y, but exclude tests with a work in process label (e.g.
wip) with the following expression:
# Launch all tests labeled 'feature-x' OR 'feature-y' mabl tests run-cloud --labels feature-x feature-y # Launch all tests with labels, but exclude WIP labeled tests mabl tests run-cloud --labels feature-x feature-y --exclude-labels wip
You can also use branches to select which test variants to run using
--mabl-branch <branch-name>. Additionally, you can run only tests that were modified on a branch (rather than the rest of the branch) with the
--branch-changes-only flag, as shown below:
# Constrain tests to a specific branch with a specific test id mabl tests run-cloud --mabl-branch my-feature-branch --id <id> # Constrain tests to only a specific branch AND only tests edited on that branch that match a label called WIP mabl tests run-cloud --mabl-branch my-feature-branch --branch-changes-only --labels WIP
To see all options for the
run-cloud command, just use the
$ mabl tests run-cloud --help Run test(s) in the mabl cloud Options: --version Show version number [boolean] -h, --help Show help [boolean] --id the id of the test to run [string] --deployment-id, -d Deployment to run the mabl tests against [string] --url, -u URL to run the mabl test against [string] --browsers, -b Space delimited browsers to test against (e.g. "chrome firefox") [array] [required] --api-key, -k API key (found in the mabl web app) [string] --mabl-branch Mabl branch to run test against [string] --branch-changes-only Only execute tests changed on specified branch [boolean] [default: false] --revision, --rev Code revision hash (application under test) [string] --labels Space delimited test labels. Run tests that match any label. [array] --exclude-labels Space delimited test labels. Exclude tests that match any label. [array] --workspace-id, -w Workspace to run against [string] --prompt Prompt to confirm execution selections [boolean] [default: true] --no-prompt, --yes Do not prompt to confirm execution selections [boolean] Examples: mabl-dev tests run-cloud --id <id> run test by id mabl-dev tests run-cloud --labels <label1> <label2> run tests by test label
To trigger a deployment event that runs tests for a given application and environment, run the following command:
mabl deployments create --application-id <application_id> --environment-id <environment_id> # Specify source control information to use with mabl GitHub App mabl deployments create \ --application-id <application_id> \ --environment-id <environment_id> \ --revision 41d502b463a37e87047148a6000411a1e1bf9a79 \ --repository-url firstname.lastname@example.org:foo-org/bar-repo.git
This will trigger a run of all mabl plans for that app environment that have been configured to run on deployment events and are not disabled. You can further narrow down which plans are run on deployment by using the
Note that passing the
--repository-url flags allow coordination between tests started with the mabl CLI and the mabl GitHub App. If you provide these values, and have the GitHub App installed, then GitHub Checks will appear with detailed tests results in your GitHub commits and pull requests. Read more about the mabl GitHub integration setup.
To get the IDs for your mabl application and environment, go to Settings --> APIs in the mabl app, choose Deployment Events API, select Application and Environment and scroll to the bottom of the page where you will see a generated CURL command showing the respective IDs.
You can see all deployment events in the mabl app by going to Results --> By Deployment.
To see all deployment events in mabl, run:
mabl deployments list
And to get details for a specific deployment, run:
mabl deployments describe <deployment_id>
For all supported deployment commands and options, run:
mabl deployments --help
You can use the mabl CLI to export all screenshots for a given test run. Customers typically use the screenshots to have product design discussions with their team or for compliance reasons. For example, if you have a test that tests the user signup and onboarding flow of your app, you can get the screenshots and discuss with your team ways to optimize the user experience to increase signups.
First you'll need the test run id. Please see the Obtaining mabl Resource IDs for the CLI section below for how to get a test run id from the test output page.
Next, go to your terminal and run the following mabl CLI command using the test run id that you copied previously:
mabl test-runs export <test-run-id>
This will package all screenshots as a ZIP file and download it to your computer in the working directory from which you run the CLI command. Note that if there are many steps in the test, downloading the screenshots may take minutes and result in files of 100MiB or larger.
To see all export options, run:
mabl test-runs export --help
To update your version of the mabl CLI tool, run:
Have another use case in mind for the mabl CLI? Please let us know by posting on our feedback portal. We love acting on user feedback.
You can get, set, and list available configuration options using the list subcommand. This command can be used to set the current workspace default in the mabl CLI amongst other options.
# view current configuration values mabl config list # set a configuration value mabl config set workspace <WORKSPACE_ID> # get a configuration value mabl config get workspace
The mabl CLI supports forward proxies. To use a proxy, set the
http.proxy configuration value to the URL for your proxy server. The URL can contain basic authentication credentials and port information.
If you receive an error that the URL you are specifying is invalid, please confirm that you are using Node version 10 or greater.
# add a proxy with no credentials on port 3128 mabl config set http.proxy http://proxy.mycompany.com:3128 # add a proxy using credentials on port 3128 mabl config set http.proxy http://"username:password"@proxy.mycompany.com:3128 # delete the current proxy setting mabl config delete http.proxy
If you receive a
407 error when logging in, this is a proxy authentication error. Ensure your password and username have been set correctly above.
Proxy settings only apply to mabl traffic
Note that if the above configuration option is set, only mabl-specific requests will go through the forwarding proxy. For local test executions, all traffic to the application under test will use the proxy settings of the browser or the operating system.
By default, the CLI verifies server certificates are issued by a known certificate authority. Certificate verification may fail if your proxy is intercepting HTTPS traffic. If you are using a proxy and SSL verification is failing, you can disable certificate verification as shown below. We suggest you do this only if required.
# disable SSL verification mabl config set http.sslVerify false # delete the current ssl verification setting to use the default mabl config delete http.sslVerify
The mabl web app offers an easy way to get the needed test IDs, test run IDs, and more for use in the mabl CLI through the mabl CLI resource IDs modal
the mabl CLI resource IDs modal
You can easily get the test ID from the test details page using the mabl CLI resource ID viewer. Click the terminal icon in the upper left toolbar on the page to open the modal. The modal will contain the test-id along with other relevant IDs to be used in the CLI. Use the copy button to easily copy the IDs.
CLI info button
The test output page also contains the CLI resource ID viewer. You can access the same modal from the actions menu in the upper right corner next to the "Edit steps" button. From the dropdown menu select "CLI info" to open the modal which will now contain the
run-id for that test output.
The actions menu location on the test output page
Updated about a month ago
|Selenium IDE export|