Skip to main content

Running tests inside the CI environment

Molly Exten
  • Updated

With the mabl CLI tool, you can run mabl tests from inside your CI/CD environment using the mabl Runner in headless mode. When orchestrating tests to run in your CI environment, you can either run them directly through mabl CLI commands or you can use the mabl-cli node package as a dependency in your project and define mabl tests with a testing framework such as Jest.

You can use mabl CLI commands directly in your CI environment just as you do locally.

If this is your first time using the mabl CLI, please refer to the CLI overview documentation for additional information.

Install the mabl CLI

Install the mabl CLI in your build machine/container with the following command:

npm install @mablhq/mabl-cli

Authenticate the mabl CLI

When running the mabl CLI in your CI environment, you’ll need to use a “Command Line Interface” mabl API key. You can obtain this key here or by navigating to the settings page in the mabl web application, then the APIs tab, and clicking the Create API key button in the upper right. From here, choose the Command Line Interface option (not the CI/CD Integration option) and provide the key a name.

 

After installing the mabl CLI in your CI, use the following command to authenticate the CLI with your mabl account using the new API key .

mabl auth activate-key KEY_VALUE

Start your web application

Before you run tests inside your CI environment, you’ll want to start your web application to ensure that the app is accessible before you run your mabl tests. If you already have this configured, then you can skip this step. You can use tools such as wait-on and wait-for-localhost-cli to perform the waiting. For instance, if you use npm start to launch the app you can use:

npm start & wait-on http://localhost:3000

Running the mabl tests

Option 1: Running tests from the command line

With the mabl CLI installed and the web app running inside the CI environment, you can now trigger your mabl tests. You can use the mabl tests command options to specify what tests you’d like to run:

Run a set of tests using a plan id

mabl tests run --from-plan-id PLAN_ID --headless

This command uses the plan information as a grouping mechanism only. It is not a plan run and advanced plan configurations (e.g. stages, shared variables) will not be adhered to.

To run a set of tests based using test labels:

mabl tests run --labels LABEL_1 LABEL_2 --exclude-labels LABEL_3 --headless

To run a specific test:

mabl tests run --id TEST_ID --headless
mabl tests run-alpha --id TEST_ID --headless

To run a test configured the same way as an existing cloud test run

mabl tests run --run-id RUN_ID --headless

Option 2: Running tests from code

Another option for kicking off mabl tests in your CI environment is to use the mabl CLI as a node package in your web app and orchestrate the mabl tests through a JavaScript testing library.

Please, refer to the JavaScript testing frameworks page for more info.

By defining your mabl tests in code you can create an npm command in your package.json file that will run the tests desired. This will require the above steps around installing, authenticating, and starting the mabl app as option 1 did. To run the tests you can either bundle them alongside your other tests or you can create a specific script for them in your package.json file

Be sure the install the mabl-cli as a dev dependency of your project with the following command:

npm install --save-dev @mablhq/mabl-cli

As an example, adding the following script to your package.json file will use Jest to call our mabl tests file:

“scripts”:
 {
    ...
    "mabl:test": "jest mablTests.test.ts",
    ...
}
import {createMablTestRunner, TestRunConfig} from '@mablhq/mabl-cli';

jest.setTimeout(180000);

const CICD_BRANCH_NAME = process.env.CI_BRANCH;

const runConfig: TestRunConfig = {
  credentialsId: 'CREDS_ID',
  environmentId: 'ENVIRONMENT_ID',
  workspaceId: 'WORKSPACE_ID',
  url: 'http://localhost:3000',
  branchName: CICD_BRANCH_NAME ?? 'master',
  headless: true, // toggle to watch tests locally
};

describe('Super important mabl tests', () => {
  it('This needs to always work', async () => {
    const mablTestRunner = await createMablTestRunner({
      testId: 'TEST_ID',
      ...runConfig,
    });
    const result = await mablTestRunner.run();
    expect(result.success).toEqual(true);
  });
});

Now you can run npm run mabl:test Inside your CI environment to kick off the mabl tests. This is also a handy command for developers to run the mabl tests on their local using the mabl CLI.

Bringing it all together

Here are some examples of what a CI configuration might look like to run mabl tests with the CLI.

# install the CLI
npm install @mablhq/mabl-cli

# authenticate with API key
mabl auth activate-key KEY_VALUE

# install wait-on to wait for the app to start
npm install wait-on

# start the web app AND wait until it loads
npm start & wait-on http://localhost:3000

# run a single mabl test
mabl tests run --id TEST_ID --environment-id ENVIRONMENT_ID --creds CREDS_ID --headless

# run a set of tests by a label
mabl tests run --labels COOL_FEATURE_ONE --environment-id ENV_ID --headless

# run a test suite of mabl tests when using option 2
npm run mabl:test

 

Related articles