Creating effective automated tests that validate the user journey can be time-consuming. With GenAI Test Creation, you can leverage generative AI to speed up the test creation process and reuse existing flows from your workspace more effectively.
This article explains how to create a test with GenAI:
Early access
During the early access period, GenAI Test Creation is available for all users to try out at no additional cost.
- Most customers: this feature is already available on the test creation page.
- Customers who opted out of AI features: if you want to try GenAI Test Creation, please reach out to your customer success manager.
- Trial users: request a demo to get access.
For transparency, mabl currently intends to include GenAI Test Creation as part of the Advanced AI add-on at general availability. Please note that this approach is subject to change.
Define the intent of the test
When you create a new browser or mobile test, include a description of what you want the test to accomplish.
Providing an intent for a test
If a test case or user story is available, use it as a starting point and augment it with an explicit description of the particular goals of the test. While it’s not necessary to describe every single click along the way, providing additional high-level details about the full end-to-end user journey can yield more successful results.
For example, you can augment the description “buy Kindle on Amazon” with more details about who the user is and how they should complete the task:
“Verify an existing Prime customer can purchase an item.
Requirements:
- Use search bar to find item
- Item must be a Kindle
- Login using existing Prime account
- Verify checkout flow is successful.”
Launch the Trainer
After describing the intent of the test, fill out the remaining information on the test creation form and click on Create test. As the Trainer launches, mabl gathers information about the flows in your workspace, and the GenAI model uses this information to create an outline of your test.
For a test created with GenAI, the Trainer loads a set of generated steps, including tasks, flows, and GenAI Assertions.
- Tasks - tasks are headers that break down the overall intent of the test into smaller goals
- Flows - if any existing flows are pertinent to the test case, mabl adds them to the relevant task.
- GenAI Assertions - each task finishes with a GenAI Assertion to validate the expected behavior once a task is complete.
A test with mabl-generated tasks
Here's what you can expect with GenAI Test Creation in the following situations:
- Empty workspace: In a workspace with no flows, the Trainer cannot add flows to your tasks.
- Missing Advanced AI add-on: If your workspace doesn't have access to the Advanced AI add-on, the Trainer won't add GenAI Assertions to the end of your tasks.
- Using the same prompt multiple times: Generative AI is non-determistic. If you repeatedly generate tests with the same prompt, you could end up with slightly different test outlines even if the core logic is similar.
- Defining the test intent in a non-English language: GenAI Test Creation is biased towards English. If you define the test intent in a non-English language, the Trainer may still generate tasks and assertions in English.
Build out the test
Review each task in the Trainer and train any additional steps required to complete the task:
Make sure your application is in the right state and the “Rec” button is on before you insert new steps.
Add steps
To add steps to a task, hover over the task and click on the Insert step button to place the Trainer cursor at that location.
Adding a step to a task
Build out flows
If the task contains a flow, you may need to train additional steps before or after the flow to accomplish the task. If the flow uses parameters, make sure the parameter values are properly configured.
To ensure mabl selects appropriate flows for your test, we recommend reviewing the existing flows in your workspace on a regular basis.
The following practices can help improve the overall quality of the flows:
- Use flow naming conventions
- Add echo steps, renamed steps, and assertions for readability
- Make sure flows target one specific purpose
- Remove redundant and unused flows
Complete GenAI Assertions
Test the GenAI Assertions to make sure they pass and generate appropriate criteria:
- Click on the edit pencil next to the GenAI Assertion.
- Revise the assertion description if it is too vague or specifies inaccurate or irrelevant details.
- Click on Test assertion to generate assertion criteria. Generating assertion criteria helps ensure the reliability of the GenAI Assertion.
For more information on working with GenAI Assertions, check out these articles:
Iterate as needed
Play back the test in the Trainer and iterate on this process as needed:
- Record any necessary steps that are not covered by the imported flows within each task.
- Revise GenAI Assertions to make sure they pass and generate the appropriate criteria.
- If necessary, update the structure of the test by deleting, adding, or editing tasks.
To create a new task header, insert an echo step and add the special symbol “-->” before the text.
Learn more
Check out this video for a demonstration on how to work with GenAI Test Creation.