If you need to validate more complex aspects of your application, try creating assertions with generative AI. With GenAI Assertions, you can validate the content of a page or a downloaded file using natural language prompts. They're easy for anyone on the team to create and maintain, and they don't require advanced knowledge of element or page attributes.
If you aren't sure when to use GenAI Assertions, this article provides guidance on when to use GenAI Assertions by highlighting common use cases and known limitations.
GenAI Assertion in a mobile test
Protecting customer data
mabl’s generative AI capabilities are built on top of Google Cloud’s enterprise AI tools. Neither mabl nor our service partner, Google Cloud, use customer data for training these models. If you have any concerns about the use of generative AI in mabl, please reach out to your customer success manager.
Use cases for GenAI Assertions
GenAI Assertions are great for evaluating the content and quality of text and images in your application, particularly content that is difficult or impossible to validate with traditional HTML or CSS assertions.
For additional guidance on crafting a prompt that yields valid and reliable results, see the article on best practices for GenAI Assertions.
Validate text content
| Use case | Sample prompt |
|---|---|
| Verify the presence or absence of certain words or phrases, including variable values. | “Verify that {{@label}} is visible in one or more rows of this table.” |
| Verify that the page is in the correct language. | “Confirm that all the text on the page is in Spanish.” |
| Check for correct grammar and spelling. | “Verify that the blog post has no grammatical errors or typos.” |
| Verify proper formatting, such as bold or hyperlinked text. | “Verify that the privacy policy is a hyperlink.” |
| Validate multiple aspects of text and visual context in one assertion | “Confirm the following are present in the screenshot: - The mabl logo - A button to start a free trial - A language picker - Also validate that there are no broken images.” |
Validate quality of dynamic content
| Use case | Sample prompt |
|---|---|
| Verify that the results are correct given a specific input. | “Confirm that the results on the page are relevant to the {{@search_input}}.” |
| Verify that the response from a chatbot is appropriate given a specific input. | “Confirm that the chatbot response is relevant to the {{@question_input}}.” |
| Verify the tone of text | “Verify that the response from the chatbot uses a professional tone.” |
| Ensure that AI-generated text is grammatically correct, factually accurate, and relevant to the topic | "Verify that this product description is grammatically correct, has no typos, and clearly describes the features and benefits of {{@product_item}} in a coherent manner." |
| Validate that personalized recommendations, advertisements, or user interfaces are displayed correctly and appropriately for a given user profile. | "Verify that the promotions displayed on this page are related to travel, such as airline deals, hotel bookings, or travel gear, and are appropriate for a user planning a vacation." |
Validate images
| Use case | Sample prompt |
|---|---|
| Check for the presence or contents of an image, such as colors or objects | “Confirm that there is an image of a globe with Greek characters in it.” |
| Verify that an image is correct given a specific input. | “Confirm that all photos show an image that is relevant to {{@search_input}}.” |
| Create moment-in-time assertions for video content. | “Verify that the bottom of the video shows a pause button, a settings cog, a volume controller, and an icon to make it full screen.” |
| Assert that a page adheres to a certain visual theme or design, even if specific elements change. | "Ensure that the search results page shows a winter-themed background." |
Limitations
Some use cases are supported by GenAI Assertions better than others. The following types of GenAI Assertions are not well supported or need additional context:
Sorted tables
If you ask that a table is sorted, the models used for GenAI Assertions have a tendency to lean toward “strict” ascending or descending order. If there are duplicate values in the table, the assertion will fail. In this situation, you should write a more explicit assertion description that specifies that the sorting can include duplicate values.
Math
Generative AI can make mistakes in assertions that require math, such as Make sure all rows add up to less than X or Make sure the value in the total column equals the sum of columns A.
Visual overlap
The GenAI model doesn’t perform well in assertions that require an understanding of overlapping or partially obscured elements, such as “Is the text on this page partially covered?”
Spatial arrangement
GenAI models don’t perform well at interpreting prompts to analyze the spatial arrangement of a page, such as whether all the images are centered or whether the items on the screen are in a grid view or a list view.
Learn more
- Creating GenAI Assertions - How to use GenAI Assertions to validate page content
- Best practices for GenAI Assertions - How to write effective assertion descriptions
- Download assertions - How to use GenAI Assertions to validate the contents of a downloaded file (web only)