Reviewing visual smoke tests

How to review output for a visual smoke test

When mabl generates visual change insights for a visual smoke test, you can follow this workflow for reviewing visual smoke test output:

  1. Identify visual changes
  2. Choose a review action
  3. Escalate flagged steps
  4. Resolve flagged steps

Identify visual change

On the Insights page, click on the "New visual changes" link for the test that you would like to review.

1207

The Insights page

Click on the RESULTS link to view test output.

1139

Visual change insights

You can filter the output to show only steps with visual changes. Steps with visual changes have an orange camera icon.

1295

Test Output page for a visual smoke test

Click on the step to compare screenshots. The screenshot tab has three views:

  • Current: screenshot from the current run
  • Baseline: screenshot from the run in which mabl established the visual baseline
  • Side-by-side: comparison view of the current and baseline screenshots

Choose a review action

For each step with visual changes, choose one of the following actions from the Review dropdown:

  1. Accept as new baseline: in future plan runs the current screenshot becomes the visual baseline.
  2. Dismiss as reviewed: maintain the existing visual baseline.
  3. Flag as visual regression: add a flag to the step to indicate that the page needs to be investigated.
984

Reviewing a detected visual change in a visual smoke test

Escalate flagged steps

Flagging a step identifies it for your team as a change that needs further review. Some options for escalating flagged steps in visual smoke tests include:

  • Share a link to the test run with your team so that they can look at the results. If you are working in the Desktop App, follow the steps on sharing links outlined here.
  • If the Jira integration is set up for your workspace, create a new issue from the Test Output page.

Resolve flagged steps

If your team reviews flagged steps for visual regressions and finds that the visual changes in the flagged step are expected, they can accept the new visual change: Review > Accept as new baseline.

If the visual changes in the flagged step indicate a visual regression, your team can fix the issue in the app and then dismiss the step in the visual smoke test as reviewed: Review > Dismiss as reviewed.


Related resources