Test snapshots

A snapshot of a mabl branch (master or any other branch) is a collection of test versions that exist in a workspace when the snapshot is created. In other words, it is a way to freeze the state of your tests and flows at a certain point in time. While development branches enable you to make isolated changes to tests, snapshots enable you to pin a consistent version of tests and flows to a mabl environment.

🚧

Snapshots become the default

If an environment is configured to use a snapshot, mabl will run the version of tests that was the latest when the snapshot was created for plan runs in that environment. In other words, mabl will use the snapshot version by default whether you run the plan on a schedule, ad hoc (by clicking on the "Run plan" button), or via the deployment events API.

Use case

The main use case for snapshots is to run a set of tests against a stable environment, where no further changes in the application are expected. Meanwhile, in other environments that are being actively updated, you can continue to develop tests and flows. As code changes are promoted across environments in the CI/CD pipeline, test changes can be promoted alongside by triggering a new snapshot.

If you want to give snapshots a try, we recommend against using a snapshot for the first environment within your deployment pipeline and instead snapshotting downstream environments. In this setup, your master branch in mabl will be running against your first environment and you can update your master without worrying about updating snapshot versions or affecting downstream environments. Then, when the code gets deployed to a downstream environment you can update the respective snapshot off of master.

16081608

Example scenario of branching and snapshotting tests as code moves through the deployment pipeline

Example scenario

The previous chart illustrates the following example scenario, where snapshots are pinned to two downstream environments, QA and Prod:

  1. A developer has made changes to the user registration flow in their app on a code branch called "user_registration_update."
  2. The developer updates the affected end-to-end tests in mabl and saves the changes to a mabl branch of the same name - "user_registration_update."
  3. The developer runs and validates the branched version of the mabl tests in her local app environment.
  4. When the code is merged to master and deployed to a Dev environment, the mabl branch is also merged to master and the updated master test version runs against the Dev environment right after the deployment and on schedule, once a night.
  5. At this moment, the code changes haven't been deployed to the two downstream environments: QA and Prod. These environments are running a snapshotted version of the tests that is older than the updated master.
    Note: If a test is not included in a snapshot, it will not run.
  6. When the code gets deployed from Dev to QA, the snapshot for the QA environment gets updated with the latest master version of the tests while Prod continues to run its older snapshot version.

📘

Beta feature

Snapshots are currently in beta and we encourage you to share your feedback via in-app chat or at productportal.mabl.com.

Currently, you can only create and update snapshots via the mabl CLI.

Creating and updating snapshots

You can create or update a snapshot by using the mabl CLI and the mabl deployments create command with the --snapshot-from option.

$ mabl deployments create -k [api-key] -p [app-id] -e [env-id] --snapshot-from [branch_name or environment_id]

To create or update the snapshot for an environment, you must specify an environment ID and an existing mabl branch name (e.g. new_feature, master, etc) or an environment ID.

  • Creating a snapshot from environment ID: The main case for this option is when you are promoting test changes from an environment with a snapshot to another environment with a snapshot and you don't want to use the latest test versions for a branch such as master.
  • Creating a snapshot from a branch: There is no need to merge a branch into master in order to snapshot its latest test version for a given environment.

Running this CLI command will create a deployment event in mabl, which will trigger a run for all active plans that match the parameters you provide. The plan runs will appear with a branch badge indicating that the test runs happened on a snapshot:

27062706

Running tests on deployment with an environment snapshot.

📘

Creating a snapshot without triggering runs

If you would like to create a snapshot for an environment but not trigger a run for the test plans associated with it, you can add a nonexistent label using the -l or --labels option. For example: mabl deployments create -w WORKSPACE-ID -e ENVIRONMENT-ID --snapshot-from master -l fake

(This command assumes that there is not a plan label called "fake" in your workspace!)

You can see all tests and flows that were snapshotted by going to Tests > Branches. Snapshots are displayed within the list of branches and indicated with status "snapshot". Snapshot names are automatically generated based on the environment ID they are created for.

11881188

List of mabl test branches and snapshots

You can also see which version of a test has been snapshotted by going to the test details and looking under the change history tab.

17821782

Test change history with versions on a branch and an environment snapshot.

Running tests for a snapshot

Once a snapshot has been created for an environment, mabl will run the respective snapshot version of the tests and flows. Thus, when you run plans on schedule, manually, or via the API, mabl will use the snapshot version by default. You can see this being indicated with the branch icon in the plan and test run results as shown below.

25742574

Manual plan run with an environment snapshot.

In recap, snapshots make it possible to merge your test changes on branch to master without worrying about affecting the same tests running against downstream environments where the code changes haven't been deployed. Those downstream environments (e.g. production) will be running the snapshotted version of the test until the environment snapshot gets updated.

🚧

Ad-hoc test runs

It is currently not possible to select a specific snapshot when running individual tests outside of a plan. The ad-hoc test run will run against a select environment using the test branch version that was selected on the test details page.

Removing a snapshot

To remove an associated snapshot with an environment, you need to take the following two steps.

  1. Create a deployment event in mabl via the CLI for the respective environment, similar to how you would create a snapshot, and do not supply the --snapshot-from option:
$ mabl deployments create -k [api-key] -p [app-id] -e [env-id]

Creating a new deployment for the environment without a snapshot tells mabl that you no longer want use a given snapshotted version of tests. Going forward mabl will resort to using the master branch version of the tests for your plan runs unless you run the plan with overrides.

  1. Go to Tests => Branches and delete the snapshot from the mabl UI so it no longer appears in the list. This step ensures that you completely remove the snapshot from the mabl UI.

Related resources