Skip to main content

Test snapshots

Eva Ferreira
  • Updated

A snapshot of a mabl branch (master or any other branch) is a collection of test versions that exist in a workspace when the snapshot is created. In other words, it is a way to freeze the state of your tests and flows at a certain point in time. While development branches enable you to make isolated changes to tests, snapshots enable you to pin a consistent version of tests and flows to a mabl environment.

Snapshots become the default

If an environment is configured to use a snapshot, mabl will run the version of tests that was the latest when the snapshot was created for plan runs in that environment. In other words, mabl will use the snapshot version by default whether you run the plan on a schedule, ad hoc (by clicking on the Run button), or via the deployment events API.

Use case

The main use case for snapshots is to run a set of tests against a stable environment, where no further changes in the application are expected. Meanwhile, in other environments that are being actively updated, you can continue to develop tests and flows. As code changes are promoted across environments in the CI/CD pipeline, test changes can be promoted alongside by triggering a new snapshot.

If you want to give snapshots a try, we recommend against using a snapshot for the first environment within your deployment pipeline and instead snapshotting downstream environments. In this setup, your master branch in mabl will be running against your first environment and you can update your master without worrying about updating snapshot versions or affecting downstream environments. Then, when the code gets deployed to a downstream environment you can update the respective snapshot off of master.

Example scenario of branching and snapshotting tests as code moves through the deployment pipeline

Example scenario of branching and snapshotting tests as code moves through the deployment pipeline

Example scenario

The previous chart illustrates the following example scenario, where snapshots are pinned to two downstream environments, QA and Prod:

  1. A developer has made changes to the user registration flow in their app on a code branch called "user_registration_update."
  2. The developer updates the affected end-to-end tests in mabl and saves the changes to a mabl branch of the same name - "user_registration_update."
  3. The developer runs and validates the branched version of the mabl tests in her local app environment.
  4. When the code is merged to master and deployed to a Dev environment, the mabl branch is also merged to master and the updated master test version runs against the Dev environment right after the deployment and on schedule, once a night.
  5. At this moment, the code changes haven't been deployed to the two downstream environments: QA and Prod. These environments are running a snapshotted version of the tests that is older than the updated master.
    Note: If a test is not included in a snapshot, it will not run.
  6. When the code gets deployed from Dev to QA, the snapshot for the QA environment gets updated with the latest master version of the tests while Prod continues to run its older snapshot version.

Creating and updating snapshots

You can create or update a snapshot by using the mabl CLI and the mabl deployments create command with the --snapshot-from option.

mabl deployments create -k [api-key] -a [app-id] -e [env-id] --snapshot-from [branch_name or environment_id]

To create or update the snapshot for an environment, you must specify an environment ID and an existing mabl branch name (e.g. new_feature, master, etc) or an environment ID.

  • Creating a snapshot from environment ID: The main case for this option is when you are promoting test changes from an environment with a snapshot to another environment with a snapshot and you don't want to use the latest test versions for a branch such as master.
  • Creating a snapshot from a branch: There is no need to merge a branch into master in order to snapshot its latest test version for a given environment.

Running this CLI command will create a deployment event in mabl, which will trigger a run for all active plans that match the parameters you provide. The plan runs will appear with a branch badge indicating that the test runs happened on a snapshot:

Running tests on deployment with an environment snapshot.

Running tests on deployment with an environment snapshot.

Creating a snapshot without triggering runs

If you would like to create a snapshot for an environment but not trigger a run for the test plans associated with it, you can add a nonexistent label using the -l or --labels option. For example: mabl deployments create -w WORKSPACE-ID -e ENVIRONMENT-ID --snapshot-from master -l fake

(This command assumes that there is not a plan label called "fake" in your workspace!)

You can see all tests and flows that were snapshotted by going to Tests > Branches. Snapshots are displayed within the list of branches and indicated with status "snapshot". Snapshot names are automatically generated based on the environment ID they are created for.

List of mabl test branches and snapshots

List of mabl test branches and snapshots

You can also see which version of a test has been snapshotted by going to the test details and looking under the change history tab.

Test change history with versions on a branch and an environment snapshot.

Test change history with versions on a branch and an environment snapshot.

Running tests for a snapshot

Once a snapshot has been created for an environment, mabl will run the respective snapshot version of the tests and flows. Thus, when you run plans on schedule, manually, or via the API, mabl will use the snapshot version by default. You can see this being indicated with the branch icon in the plan and test run results as shown below.

Manual plan run with an environment snapshot.

Manual plan run with an environment snapshot.

In recap, snapshots make it possible to merge your test changes on branch to master without worrying about affecting the same tests running against downstream environments where the code changes haven't been deployed. Those downstream environments (e.g. production) will be running the snapshotted version of the test until the environment snapshot gets updated.

Ad-hoc test runs

It is currently not possible to select a specific snapshot when running individual tests outside of a plan. The ad-hoc test run will run against a select environment using the test branch version that was selected on the test details page.

Removing a snapshot

To remove a snapshot, take the following two steps:

  1. Disassociate the snapshot from the environment: create a deployment event in the mabl CLI for the environment without the --snapshot-from option. For example:
mabl deployments create -k [api-key] -a [app-id] -e [env-id]
  1. Remove the snapshot from the mabl UI: Go to Tests > Branches in the mabl app and delete the snapshot from the mabl UI so it no longer appears in the list.

Going forward mabl resorts to using the master branch version of the tests for your plan runs unless you specify a different branch.

If you disassociated a snapshot from an environment by mistake, you can run this command to pin the environment to the snapshot again:

mabl environments update [env-id] --mabl-branch [snapshot-name]

Checking an environment's snapshot status

To check whether an environment is configured to use a snapshot, run the following command in the mabl CLI:

mabl environments describe [env-id]

The mabl CLI returns metadata on the environment, including whether or not it is configured to use a snapshot:

  • If the metadata shows use_source_control_tag: true, the environment is configured to use a snapshot.
  • If the metadata shows use_source_control_tag: false, or the use_source_control_tag property is absent, the environment is not configured to use a snapshot.

Related articles