Test snapshots

A snapshot of a mabl branch (master or any other branch) is the collection of test versions that exist in a workspace when the snapshot is created. It is a way to freeze the state of your tests at a certain point in time.

While test branches enable you to make isolated changes to tests, snapshots enable you to pin specific test changes to a mabl environment. If an environment is configured to use a snapshot, mabl will run the version of tests that was the latest when the snapshot was created for any plan runs against that environment.

Use case

The main use case for snapshots is to enable you to update tests due to code changes deployed to one environment (e.g. dev) without affecting the tests running against other environments (e.g. staging, production) where the code changes haven't been yet deployed. Thus, you can promote test changes together with code changes across environments in your CI/CD pipeline, or traditional deployment workflow.

If you have integrated mabl into your CI/CD pipeline, you should give snapshots a try. We also recommend that you don't use a snapshot for the first environment within your deployment pipeline and instead snapshot all downstream environments. In this setup, your master branch in mabl will be running against your first environment and you can easily update your master without worrying about updating snapshot versions or affecting downstream environments. Then, when the code gets deployed to a downstream environment you can update the respective snapshot off of master.

Example scenario of branching and snapshotting tests as code moves through the deployment pipeline.

Example scenario of branching and snapshotting tests as code moves through the deployment pipeline.

Example scenario

The above chart illustrates the following example scenario. Developer has made changes to the user registration flow in the app under test on a code branch user_registration_update. Developer updates the affected end-to-end tests in mabl and save the changes to a mabl branch. She can run and validate the branched version of the mabl test in her local app environment using the mabl CLI or the Trainer. Then, when the code is merged to master and deployed to a Dev environment, the mabl branch is also merged to master and the updated master test version runs against the Dev environment right after the deployment and on schedule, once a night. At this moment in time, the two downstream environments, QA and Prod, are running a snapshotted version of the tests that is older than the updated master since the code changes haven't been deployed to those environment yet. When the code gets deployed from Dev to QA, the snapshot for the QA environment gets updated with the latest master version of the tests while Prod continues to run an older snapshot version.

Beta feature

Snapshots are currently in beta and we encourage you to share your feedback via in-app chat or at feedback.mabl.com.

Currently, you can only create and update snapshots via the mabl CLI.

Creating snapshots

You can create a snapshot today by using the mabl CLI and the mabl deployments create command with the --snapshot-from option.

$ mabl deployments create -k [api-key] -p [app-id] -e [env-id] --snapshot-from [branch_name or environment_id]

To create or update the snapshot for an environment, you must specify an environment ID and an existing mabl branch name (e.g. new_feature, master, etc) or an environment ID. The main case for creating a snapshot from an environment ID is when you are promoting test changes from an environment with a snapshot to another environment with a snapshot and you don't want to use the latest test versions for a branch such as master.

Running the above CLI command will create a deployment event in mabl that will trigger the respective test plan runs and the you will see a branch badge indicating that the test runs happened on a snapshot as shown below.

Running tests on deployment with an environment snapshot.

Running tests on deployment with an environment snapshot.

There is no need to merge a branch into master in order to snapshot its latest test version for a given environment.

You can see all tests and flows that were snapshotted by going to Tests > Branches. The snapshots are displayed within the list of branches and indicated with status "snapshot". Snapshot names are automatically generated based on the environment ID they are created for and the word SNAPSHOT appended to it as show below.

List of mabl test branches and snapshots.

List of mabl test branches and snapshots.

You can also see which version of a test has been snapshotted by going to the the test details and looking under the change history tab.

Test change history with versions on a branch and an environment snapshot.

Test change history with versions on a branch and an environment snapshot.

Running tests for a snapshot

Once a snapshot has been created for an environment, mabl will run the respective snapshot version of the tests and flows. Thus, when you run plans on schedule, manually, or via the API, mabl will use the snapshot version by default. You can see this being indicated with the branch icon in the plan and test run results as shown below.

Manual plan run with an environment snapshot.

Manual plan run with an environment snapshot.

In recap, snapshots make it possible to merge your test changes on branch to master without worrying about affecting the same tests running against downstream environments where the code changes haven't been deployed. Those downstream environments (e.g. production) will be running the snapshotted version of the test until the environment snapshot gets updated.

Ad-hoc test runs

It is currently not possible to select a specific snapshot when running individual tests outside of a plan. The ad-hoc test run will run against a select environment using the test branch version that was selected on the test details page.

Removing a snapshot

To remove an associated snapshot with an environment, you need to take the following two steps.

First, create a deployment event in mabl via the CLI for the respective environment, similar to how you would create a snapshot, and do not supply the --snapshot-from option:

$ mabl deployments create -k [api-key] -p [app-id] -e [env-id]

Creating a new deployment for the environment without a snapshot tells mabl that you no longer want use a given snapshotted version of tests. Going forward mabl will resort to using the master branch version of the tests for your plan runs unless you run the plan with overrides.

There's one more step you need to take to complete remove the snapshot from the mabl UI. Go to Tests => Branches and delete the snapshot from the mabl UI so it no longer appears in the list.

Happy testing!

Updated 2 months ago

Test snapshots

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.