You can improve your test coverage and start seeing results in less than five minutes! .
Visual smoke test results overview
A visual smoke test differs from the other mabl tests in the following ways:
- Detects visual changes for each browser the test runs in while other tests only do so in Chrome.
- Checks for broken links on each of the specified pages instead of crawling everything
- Offers a more elaborate workflow to review visual changes, update the baseline and flag regressions for each configured browser type.
Note that having broken links is not only bad for the user experience but it can also hurt your rankings in search engine results (e.g. Google, Bing), especially for your marketing website which should be an essential part of your app and the "whole product experience."
The visual smoke test is great for continuous testing across your staging and production environments to validate:
- Key pages from the customer's journey on your marketing website, such as the home page, product overview and pricing
- Key landing pages for your marketing website, such as trial signup, demo request, and whitepaper download that you drive traffic through via advertisement
- Key pages of your web application, such as signup, login, user settings, etc.
- Identify key pages for your web app and site based on your analytics
- Create a visual smoke test from the mabl app for those page URLs
- Run that test manually, on schedule and/or deployment event
- Review test results and accept, dismiss or flag visual changes
- Capture and address any issues with the app or website under test
Creating a visual smoke test is simple, and it doesn't require using the mabl Trainer. You can click to create a new visual smoke test from two places:
- The "New" button on the Dashboard page
- The "New test" button options on the Tests page
You will then be presented with a form where you can enter up to 50 page URLs for mabl to test under a given testing plan. Fill out the form to create the test.
Easily create lists of URLs
In Chrome's Bookmark Manager, you can multi-select bookmarks and copy them and then paste them into your test!
Visual smoke test creation form.
Once you complete the test creation form, you will see the list of page URLs you specified with a mabl-generated description of the test steps. You can edit the page URLs and other test properties, such as description and labels, but you cannot edit the test steps since those are auto-generated for the test based on mabl algorithms.
Visual smoke test details.
To run the test, you can use the "Run test" button on the test details page or run it as part of a plan just like all other tests in mabl. When you run the test for the first time, mabl will visit each page URL to:
- Check it loads properly (200 status code response)
- Capture a screenshot of the page to establish a fixed visual baseline
- Capture a DOM snapshot with all page network activity for diagnostic purposes
To start seeing visual changes, you need to run a test at least two times since the first run is used to establish a visual baseline. This requirement also applies when you update the page URLs because mabl assumes it needs to create a new visual baseline for the updated URLs.
You will receive the following insights as a result of a visual smoke test:
- Visual change detected
- Broken links
Those insights will show up in the mabl app under the Insights section as well as in Slack, if you have it configured.
Visual smoke test limitations
- The viewport cannot be changed and is currently set to 1366 x 3072 for all browsers
- Visual learning to identify dynamic content regions is not yet supported for this type of test
- It is not yet possible to manually specify regions to ignore for visual change detection
If you would like us to prioritize addressing these limitations, or want to request additional features, please let us know at feedback.mabl.com.
When viewing results for the visual smoke test, you will see thumbnails of all page screenshots to the left with the full screenshot of the selected page to the right. The number of detected visual changes and broken links is listed at the top.
Visual smoke test results.
You will see a yellow camera icon on the thumbnail for each page with visual changes. Selecting it, will allow you to examine the changes in a side-by-side view and take one of the following actions on the current screenshot with visual changes:
- Accept it as the new baseline for future test runs
- Dismiss it as reviewed without accepting as a new baseline or flagging it as an issue
- Flag it as visual regression so it can be fixed
When you flag a visual change as regression, that flag will remain as part of the test run and you can simply share a link to the test run with your software development team so they can take a look at the results. You can also capture regression issues in Jira directly from the test results page, if you have the integration configured.
To get the list of broken links, simply click on the number of broken links and download the report as a CSV file.
Updated 6 months ago