Visual smoke test output

Understanding the test output of a visual smoke test

When a visual smoke test runs in the cloud, the test output includes the following:

  • Screenshots and logs: Similar to browser test output, visual smoke tests update the Test Output page with screenshots and logs in real time.
  • Visual changes: Mabl processes each screenshot to detect visual changes. If mabl detects a visual change for a particular page in the test, the visual change appears on the Test Output page.

While mabl is still calculating visual changes, a banner appears on the Test Output page: Scanning for visual changes; refresh for latest results.

1268

Visual smoke test output

📘

Running visual smoke tests

When a visual smoke test runs in the cloud, it counts as one browser test run.

Screenshots and logs

Every time a visual smoke test runs in the cloud, the mabl-generated flow "Visual Smoke Test" visits each URL and completes the following actions:

  • Capture a screenshot of the page
  • Collect test diagnostics:
    • Check that the page returns a 200 response
    • Capture network activity on the page
    • Capture a DOM snapshot

Visual changes

It takes two runs for mabl to identify visual changes in a visual smoke test.

  • On the first plan run, mabl establishes a visual baseline.
  • On subsequent runs, mabl compares each page to its visual baseline.

The comparison to the visual baseline is a static comparison. If the page in the current run is not identical to its visual baseline, mabl does the following:

  • Highlight the regions of the page that appear different
  • Flag the step as a visual change in the test output
  • Report a visual change insight in your workspace

📘

Updating visual smoke tests

If you make any updates to the list of URLs to visit, mabl establishes new visual baselines for all pages on the next plan run.


Related resources