Creating GenAI Assertions

With GenAI assertions, you can validate more complex aspects of your application, such as text content, quality, and images. This article explains how to create a GenAI Assertion, including tips for selecting elements, working with viewport sizes, and returning results in non-English languages. 

How to create a GenAI Assertion

To create a GenAI Assertion, open your web or mobile application in the mabl Trainer and get your application in the desired state. Then take the following steps:

1. In the Trainer window, click on the check mark icon to create a new assertion.
2. Identify what you want to validate:
   • To validate the entire page, click on GenAI Screen Capture in the Trainer window.
   • To validate part of the page, select the element that you want to validate in the browser window and select AI in the assertion menu.
3. In the Assertion description field, describe what you want to verify using natural language. Your assertion description may include references to variables used in the test, including values from DataTables and other data-driven variables. For example, "Verify that the results are relevant to {{@search_input}}."
4. Click on Test assertion. The first time you test the assertion, mabl generates assertion criteria.  This criteria is sent with the assertion description to the LLM to provide a consistent set of criteria to evaluate as part of your assertion. If you iterate on the assertion description, mabl updates the assertion criteria to reflect the current state of the description.
5. Choose a behavior on failure. By default, AI assertions are set to "Fail at end of test."
6. Click OK.

Selecting from overlapping elements

When you select an element for a GenAI Assertion, the mabl Trainer highlights the element with the highest z-index at your mouse position by default.

If you want to target a different overlapping element, hold [SHIFT] and use the up and down arrow keys to highlight different elements at your mouse position. Once the Trainer highlights the correct element, release the [SHIFT] key and click on the element.

Iterating through overlapping elements

GenAI Assertions and viewport size

Unlike HTML and CSS assertions, GenAI Assertions evaluate based on the screenshot of the element, not the DOM or the page source.

• For a GenAI Screen Capture assertion, the screenshot only includes the portion of the page that is currently visible in the screenshot. 
• For a GenAI Assertion on a specific element, the screenshot only includes the portion of the element that is visible in the viewport.

If you assert on something that isn't fully visible in the viewport, such as a long table element, the off-screen part of the element will not be evaluated.

To resize the window in a browser test, add a "set viewport" step before the assertion. The "set viewport" step ensures that the test uses the new viewport size for the GenAI Assertion during test execution. If you manually resize the browser window during a training session, the browser will still use the size specified in the "set viewport" step when run in the cloud.

Returning output in non-English languages

The models used for GenAI Assertions are biased towards returning results in English. If you submit a prompt in a non-English language, you can increase the likelihood of returning the result in the same language by specifying the desired output language.

For example, to return results in Japanese, add the sentence, "Return the results in Japanese" to the end of the prompt.