Running tests inside a CI environment

With the mabl CLI, you can run mabl tests in headless mode inside your CI environment. This execution environment is known as the mabl CI Runner. Tests execute in your CI environment sequentially and return pass/fail feedback.

Integrating the CI Runner into your development workflow is a great way to get quick validation and catch issues earlier in development. Because tests execute sequentially, the CI Runner is best suited for validating a focused set of tests rather than executing large, comprehensive suites. This article explains how to run tests in your CI environment with the mabl CI Runner:

• Install the mabl CLI
• Authenticate the mabl CLI
• Start up your app
• Run mabl tests

Install the mabl CLI

There are two ways you can install the mabl CLI in your CI environment:

• Global installation
• Local installation (recommended)

Global installation

Run npm install -g @mablhq/mabl-cli to install the mabl CLI globally in your CI environment.

After installation, you can run mabl CLI commands directly from your terminal the using this template:

mabl [command] [subcommand]
# Examples
mabl auth activate-key [api-key]
mabl tests run --id [test-id]

Local installation (recommended)

Run npm install --save-dev @mablhq/mabl-cli to install the mabl CLI as a dev dependency in your project’s package.json.

After installation, you can run mabl CLI commands from your terminal using this template:

npx @mablhq/mabl-cli [command] [subcommand]
# Examples
npx @mablhq/mabl-cli auth activate-key [api-key]
npx @mablhq/mabl-cli tests run --id [test-id]

Authenticate the mabl CLI

To run mabl CLI commands in your CI environment, you need to authenticate with a mabl API key:

1. Go to the APIs page in the mabl app: Settings > API
2. Click on + Create API key. Creating API keys is limited to workspace owners.
3. Select “Command Line Interface” as the type of API key.
4. Give the API key a name.
5. Save your settings.

In your CI environment, run the following command to authenticate the mabl CLI, replacing [mabl_api_key] with the API key you just created:

mabl auth activate-key [mabl_api_key]

If you installed the mabl CLI as a dev dependency, use npx @mablhq/mabl-cli instead of mabl in your command:

npx @mablhq/mabl-cli auth activate-key [mabl_api_key]

Start up your app

Before you run tests inside your CI environment, confirm that your web application is accessible. For example, if you use npm start to launch your app, you can use tools such as wait-on and wait-for-localhost-cli to perform the waiting.

npm start & wait-on http://localhost:3000

Note the base URL of your application within the CI environment. This is the URL your mabl tests will use during CI runs. Keep in mind that this URL might differ from the one you use to access your application locally.

Run mabl tests

With the mabl CLI installed and the web app running inside the CI environment, you can now trigger mabl tests on the CI Runner. Use one of the following methods:

• Run tests from the command line
• Run tests from code

Run tests from the command line

Use the mabl tests run command to trigger tests from the command line. If you installed the mabl CLI as a project dev dependency, remember to use npx @mablhq/mabl-cli instead of mabl in your commands.

# Run a specific test
mabl tests run --id [test-id] --url [app-url-in-ci] -e [env-id] --headless

# Run a group of tests that match the label "wip"
mabl tests run --labels wip --url [app-url-in-ci] -e [env-id] --headless

# Run a group of tests associated with a specific plan
mabl tests run --from-plan-id [plan-id] --url [app-url-in-ci] -a [app-id] -e [env-id] --headless

Please note that running mabl tests --from-plan-id in your CI environment is not the same as a plan run in the cloud:

• Advanced plan configurations, such as stages, browser settings or shared variables, are not applied.
• Tests run sequentially in the same order they appear on the plan details page in the mabl app.
• All tests use the same base URL, which is the URL associated with the plan’s application and environment. If the plan is configured with multiple URLs, the mabl CLI will use one of those URLs.
• If you include the --url option, all tests in the group use this URL.

Run tests from code

If you installed the mabl CLI as a dev dependency in your project, you can also trigger mabl tests from code. This option requires a JavaScript testing framework, such as Jest, to run your mabl tests.

The first step is to write a script that orchestrates your mabl tests. For example, if you use Jest in a TypeScript environment, create a .ts file that imports the mabl CLI dependency, configures the test runner, and triggers mabl test execution:

// Import the necessary functions from the @mablhq/mabl-cli dependency
import {createBrowserTestsRunner, TestRunConfig} from '@mablhq/mabl-cli';

jest.setTimeout(180000);

// Update placeholders for workspaceId, url, and labelsInclude
const runConfig: TestRunConfig = {
    workspaceId: 'workspace_id',
    url: 'app-url-in-ci',
    labelsInclude: ['test-label'],
    headless: true, 
    filterHttpRequests: false, 
};

// Trigger the mabl test execution
describe('Super important mabl tests', () => {
 it('This needs to always work', async () => {
   const mablTestRunner = await createBrowserTestsRunner({
     ...runConfig,
   });
   const result = await mablTestRunner.run();
   expect(result.success).toEqual(true);
 });
});

After creating the script, add it to the "scripts" section of your package.json file.

Use a descriptive name that indicates the purpose of the script, such as mabl-test. Set the value of this script to the command that executes your mabl tests. For example, if you’re using Jest, the command could be "jest mablTests.test.ts".

“scripts”:
{
   ...
   "mabl-test": "jest mablTests.test.ts",
   ...
}

Finally, use npm run or yarn run to trigger mabl tests inside your CI environment. For example, if the name of your script is mabl-test, run npm run mabl-test or yarn run mabl-test.

npm run [name-of-your-script]
# OR
yarn run [name-of-your-script]