We are pleased to share that visual assertions are now generally available! Visual assertions are great for validating more complex aspects of your web or mobile application, such as text content, linguistic analysis, and visual details.
Visual assertions promotional program
Through the end of 2024, customers and trial users can run tests with visual assertions as part of a special promotional program:
- Customers may execute up to 1000 cloud runs containing visual assertions.
- Trial users may execute up to 100 cloud runs containing visual assertions.
If the limit is exceeded before 12/31/24 or the promotional program has ended without the inclusion of the feature in your mabl plan, the feature will be disabled. Any tests or plans containing visual assertions cannot be run until those steps are removed. If the test is executed as part of a deployment event, the test containing visual assertions will be skipped.
Try it out
To try it out for yourself, launch your web or mobile application in the mabl Trainer and follow these steps to create a visual assertion.
Since introducing visual assertions in June, we've updated the behavior in the Trainer to include assertion criteria. This assertion criteria is sent with the assertion description to the LLM to provide a consistent set of criteria to evaluate as part of your assertion. If you iterate on the assertion description, mabl updates the assertion criteria to reflect the current state of the description.
With the GA release, visual assertions now capture explicit criteria in the assertion step for greater consistency. These newly added or edited visual assertions are not compatible with older versions of the Desktop App. Please make sure that your team updates to mabl Desktop App version 2.18.0 or higher to continue working with any tests that include visual assertions.
Execution limits
Visual assertions are unlimited during training so that you can iterate on your assertion as much as you need. During execution, however, keep in mind the following limitations:
- Cloud runs: visual assertions are limited to 6 per test. If they exceed this limit, the test will fail.
- CI Runner and CLI local runs: Any visual assertions will fail when run locally in the mabl CLI or the CI Runner.
- Performance tests: visual assertions will fail when run within the context of a performance test.
Learn more
To learn more about visual assertions, check out the following articles:
Protecting customer data
mabl’s generative AI capabilities are built on top of Google Cloud’s enterprise AI tools. Neither mabl nor our service partner, Google Cloud, use customer data for training these models. If you have any concerns about the use of generative AI in mabl, please reach out to your customer success manager.