Visual change detection for browser tests

When a browser test runs as part of a plan, mabl checks for visual changes in the application UI. Visual change detection is useful for monitoring your app and ensuring there are no issues with the overall appearance.

Visual change detection is managed in the edit plan page in Advanced options using the visual change learning toggle.

Manage visual change detection with the visual change learning toggle

This guide explains the two modes for visual change detection:

• Default visual comparison: when visual change learning is toggled off, mabl compares screenshots in the current plan run to the same screenshots in the previous plan run in the same environment.
• Visual change learning: when visual change learning is toggled on, mabl compares screenshots in the current plan run to a dynamic visual model created over many plan runs in the same environment.

Default visual comparison

When visual change learning is turned off, mabl compares screenshots from the current plan run to the previous plan run in the same environment.

The default visual comparison starts at the pixel level.  Each pixel is compared and considered a match if the exact same color is in the the exact pixel location of both screenshots. Then, differences are clustered and related to the full DOM information captured during the runs. Finally, components encompassing regions of individual pixel changes are identified and marked as visual changes.

If the differences between the two screenshots meet the minimum visual change threshold, which is roughly the size of a blinking cursor, mabl does the following:

• Highlight the regions of the page that appear different.
• Include a baseline and side-by-side comparison. The baseline screenshot is from the previous plan run in that environment.
• Flag the step with a “visual change” label on the test output page.

Visual change learning

If you enable visual change learning on the edit plan page, mabl starts creating a dynamic visual model.

Dynamic visual model

mabl uses machine learning to create a dynamic visual model of your application UI that ignores areas with dynamic content, such as banner ads, product listings, videos, and animations.

If mabl detects a visual change in the current plan run compared to the dynamic visual model, mabl does the following:

• Highlight the regions of the page that appear different
• Include a baseline and side-by-side comparison to the dynamic visual baseline
• Flag the step with a "visual change" label on the test output page
• Generate a visual change insight

Highlighting visual changes and dynamic content on the test output page

Building a dynamic visual model

To build a dynamic visual model for a browser test, the following conditions must be met:

• The test passed 10-12 times when run in the plan: ad-hoc test runs do not contribute to the dynamic visual model.
• The passing plan runs were in the same environment: dynamic visual models are created per environment.
• The passing plan runs occurred within 10 days: dynamic visual models require a frequency of 10-12 passing runs within 10 days.
• The plan was run on Chrome or Edge: dynamic visual models are created per browser.
• The test runs are from the same test version: every time you update a browser test, its version changes and the dynamic visual model is reset.

After these criteria are met, it can take from 30 minutes to 24 hours to build the dynamic visual model. While the test is still collecting data or building the dynamic visual model, mabl uses the default visual comparison: current run to previous run.