Test inside the CI environment

With the mabl CLI tool, you can run mabl tests from inside your CI/CD environment using the mabl Runner in headless mode. When orchestrating tests to run in your CI environment, you can either run them directly through mabl CLI commands or you can use the mabl-cli node package as a dependency in your project and define mabl tests with a testing framework such as Jest.

Early access program

The mabl Runner is currently available to select customers under the mabl Early Access Program. Request access.

You can use mabl CLI commands directly in your CI environment just as you do locally.

If this is your first time using the mabl CLI, please refer to the CLI overview documentation for additional information.

Install the mabl CLI

Install the mabl CLI in your build machine/container with the following command:

npm install @mablhq/mabl-cli

Authenticate the mabl CLI

When running the mabl CLI in your CI environment, you’ll need to use a “Command Line Interface” mabl API key. You can obtain this key here or by navigating to the Settings page in the mabl web application, then the APIs tab, and clicking the Create API key button in the upper right. From here, choose the Command Line Interface option (not the CI/CD Integration option) and provide the key a name.

After installing the mabl CLI in your CI, use the following command to authenticate the CLI with your mabl account using the new API key .

mabl auth activate-key KEY_VALUE

Start your web application

Before you run tests inside your CI environment, you’ll want to start your web application to ensure that the app is accessible before you run your mabl tests. If you already have this configured, then you can skip this step. You can use tools such as wait-on and wait-for-localhost-cli to perform the waiting. For instance, if you use npm start to launch the app you can use:

npm start & wait-on http://localhost:3000

Running the mabl tests

Option 1: Running tests from the command line

With the mabl-cli installed and the web app running inside the CI environment, you can now trigger your mabl tests. You can use the mabl tests command options to specify what tests you’d like to run:

Run a set of tests using a plan id

mabl tests run-alpha --from-plan-id PLAN_ID --headless

This command uses the plan information as a grouping mechanism only. It is not a plan run and advanced plan configurations (e.g. stages, shared variables) will not be adhered to.

To run a set of tests based using test labels:

mabl tests run-alpha --labels LABEL_1,LABEL_2 --exclude-labels LABEL_3 --headless

To run a specific test:

mabl tests run-alpha --id TEST_ID --headless
mabl tests run-alpha --id TEST_ID --headless

To run a test configured the same way as an existing cloud test run

mabl tests run-alpha --run-id RUN_ID --headless

Some of the additional options for the run-alpha command are:

--environment-id flag to specify the mabl environment to run the test under. Specifying the environment allows mabl to use the appropriate environment variables and learned find information when running the test.
--url flag to specify a particular URL to run the test against.
--mabl-branch flag to specify a mabl branch to run the test under.
--credentials flag to specify a testing credential set to use during the local run (`mabl credentials list` to see available ones).
--from-plan-id flag to get a list of tests from a plan.

Option 2: Running tests from code

Another option for kicking off mabl tests in your CI environment is to use the mabl CLI as a node package in your web app and orchestrate the mabl tests through a JavaScript testing library.

Please, refer to the JavaScript testing frameworks page for more info.

By defining your mabl tests in code you can create an npm command in your package.json file that will run the tests desired. This will require the above steps around installing, authenticating, and starting the mabl app as option 1 did. To run the tests you can either bundle them alongside your other tests or you can create a specific script for them in your package.json file

Be sure the install the mabl-cli as a dev dependency of your project with the following command:

npm install --save-dev @mablhq/mabl-cli

As an example, adding the following script to your package.json file will use Jest to call our mabl tests file:

“scripts”: {
    ...
    "mabl:test": "jest mablTests.test.ts",
    .
}
import {createMablTestRunner, TestRunConfig} from '@mablhq/mabl-cli';

jest.setTimeout(180000);

const CICD_BRANCH_NAME = process.env.CI_BRANCH;

const runConfig: TestRunConfig = {
  credentialsId: 'CREDS_ID',
  environmentId: 'ENVIRONMENT_ID',
  workspaceId: 'WORKSPACE_ID',
  url: 'http://localhost:3000',
  branchName: CICD_BRANCH_NAME ?? 'master',
  headless: true, // toggle to watch tests locally
};

describe('Super important mabl tests', () => {
  it('This needs to always work', async () => {
    const mablTestRunner = await createMablTestRunner({
      testId: 'TEST_ID',
      ...runConfig,
    });
    const result = await mablTestRunner.run();
    expect(result.success).toEqual(true);
  });
});

Now you can run npm run mabl:test Inside your CI environment to kick off the mabl tests. This is also a handy command for developers to run the mabl tests on their local using the mabl CLI.

Bringing it all together

Here are some examples of what a CI configuration might look like to run mabl tests with the CLI.

# install the CLI
npm install @mablhq/mabl-cli

# authenticate with API key
mabl auth activate-key KEY_VALUE

# install wait-on to wait for the app to start
npm install wait-on

# start the web app AND wait until it loads
npm start & wait-on http://localhost:3000

# run a single mabl test
mabl tests run-alpha --id TEST_ID --environment-id ENVIRONMENT_ID --credentials CREDS_ID --headless

# run a set of tests by a label
mabl tests run-alpha --labels COOL_FEATURE_ONE --environment-id ENV_ID --headless

# run a test suite of mabl tests when using option 2
npm run mabl:test

Updated 3 months ago


Test inside the CI environment


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.