Skip to main content

2026/01/14 - GenAI Assertions in local CLI runs

Molly Exten
  • Updated

Starting with mabl CLI version 2.82.10, you can trigger local and CI runs of tests that contain GenAI Assertions. With this enhancement, you can run more complex visual validations earlier in the development cycle!

To enable GenAI assertions for your local browser or mobile test runs, use the --allow-billable-features option. For example:

mabl tests run --id {test-id} --allow-billable-features

Promotional period

For the foreseeable future, local CLI and CI executions can run GenAI assertions at no additional credit cost.

When the --allow-billable-features option is enabled, local and CI runs output the assertion criteria and resulting analysis and evaluation from GenAI Assertion steps. Test runs are subject to the same 30-assertion-per-run limit as cloud runs. If this limit is exceeded, any additional GenAI Assertions will fail.

sample_cli_output_with_genai_assertion.png

By default, the allow-billable-features option is disabled. When disabled, GenAI Assertions are failed with the following message: "AI assertions are not available in CLI runs by default. Use –allow-billable-features to enable, failing assertion."

Try it out

To try it out, update the mabl CLI to version 2.82.10 or later:

npm install -g @mablhq/mabl-cli

To trigger a local or CI run of browser tests with GenAI Assertions, use the mabl tests run command. For example:

mabl tests run 
  --labels smoke 
  --allow-billable-features

To trigger a local or CI run of mobile tests with GenAI Assertions, use the mabl tests run-mobile command. For example:

mabl tests run-mobile 
  --labels wip 
  --build-file-id {build-file-id} 
  --allow-billable-features

Other ways to run tests locally with GenAI Assertions:

  • mabl Desktop App - trigger a local ad hoc run from the Desktop App.
  • mabl MCP - for local MCP runs with #run_mabl_test_local, you can enable GenAI Assertions with the allowLocalTestBilling: true flag. Note that while local test runs triggered from the MCP return a test status, they do not output step level results, such as the detailed output and evaluation of GenAI Assertion steps.

Related articles