We are thrilled to share that visual assertions are now available for mobile tests in early access!
Use visual assertions to expand your test coverage and validate more complex aspects of your mobile application, particularly textual and visual details that aren't easily validated using the page source.
Early access feature
Visual assertions are currently available for all mabl customers as an early access program. Trial users can reach out to the mabl team if interested in trying it out.
Prior to general availability, changes to the feature may be made without notice.
How it works
A visual assertion is a type of assertion that sends a screenshot of the target element and a description of what you want to validate to a large language model (LLM). The LLM determines whether the assertion should pass or fail in the current situation and returns a assertion result with an explanation of why the assertion passed or failed.
Try it out
If you have already opted in to visual assertions for browser tests, you will automatically be opted in to visual assertions for mobile tests.
If your team hasn't yet opted in, an your workspace can enable visual assertions on the Labs page: Settings > Labs. They may need to update to the latest version of the mabl Desktop App to see the option on the labs page.
To create a visual assertion, open your mobile application in the mabl Trainer and get your application in the desired state. Then take the following steps:
- In the Trainer window, click on the check mark icon to create a new assertion.
- Select the element on the page that you want to validate.
- Click on the AI tab in the assertion menu.
- In the Assertion description field, describe what you want to verify.
- Click on Test assertion.
- Iterate on the assertion description until you are confident that the assertion yields the desired result.
For more guidance on creating successful visual assertions, see our article on creating assertions with generative AI.
Scrolling and viewports
Visual assertions evaluted based on the screenshot of the element, not the page source. If needed, add a scroll to element step to make sure the screen shows the correct area. The screenshot included with a visual assertion only includes the portion of the element that is currently visible in the viewport.
Limitations
Visual assertions are not supported for:
- Local test runs triggered in the mabl CLI
- Tests that execute as part of a performance test run.
The models are biased toward prompts written in English. Prompts written in other languages are not guaranteed to yield a result in the same language.
During the early access period, keep in mind the following limitations:
- Visual assertions are limited to 3 per cloud run.
- If you're running plans or deployments in parallel, no more than 20 of the parallel test runs should include visual assertions. Exceeding this limit may result in the feature being disabled for the workspace.
To avoid exceeding these limits, we recommend against using visual assertions in login flows, loops, and other commonly used flows.
Generative AI at mabl
Visual assertions are built on top of Google Cloud's AI tools. Neither mabl nor our service partner, Google Cloud, use customer data for training these models. If you have any concerns about the use of generative AI in mabl, please reach out to your customer success manager.
For more information about how mabl handles customer data, see the article on data encryption.