How to write an effective test prompt

To create mabl tests with generative AI, you need to provide a test prompt that describes what you want to validate. If your test prompt is unclear, the model may suggest or perform irrelevant test steps in the Trainer that don’t match your goals.

To save time and get the most out of the Test Creation Agent, use the following guidelines for writing an effective prompt for browser and mobile tests.

Browser tests

When you create a browser test with Test Creation Agent, the agent uses your provided prompt to build out the test in the Trainer. To increase the likelihood that the agent creates a test that matches your prompt, use the following guidelines.

Give clear, discrete instructions

Describe the test case as if you were telling a manual tester how to validate the workflow. If the high-level goal of the test is “Purchase a paintbrush”, include discrete instructions on how to interact with the app:

“Search for a paintbrush using the search bar. Select the first matching item. Verify that the item detail page shows more than one image for the item. Perform the checkout process with the following shipping info:”100 Maple St. Boston, MA 02111.”

Detailed instructions can be especially helpful if the test interacts with a confusing area of your app. For example, if you need the test to navigate to a difficult to find page, include additional details on that portion of the test: “Navigate to the user settings by first clicking on configuration, then in the sub menu clicking, then settings.”

The agent will automatically import relevant flows. If you want to increase the likelihood that a specific flow gets imported, call it out in the prompt. Just be sure to indicate any prerequisite steps before running the flow: “Use the flow”Logout - App” after updating contact information.”

Call out important validations

The agent automatically adds in GenAI Assertions to validate that your app is working as expected. If you don’t include explicit instructions about what you want to validate, the agent adds assertions based on what the model thinks is important. For example, after adding an item to a cart, it’s common for the agent to assert that the item has successfully been added to the cart.

If there is something that you explicitly need to validate on the page, include it in the prompt. For example, if you want the test to validate the format of an order confirmation page, call it out in the prompt: “Verify the order confirmation includes the customer’s name and item selected”

Explain how to use associated resources

The agent has access to all test variables, including any associated DataTables or credentials. If you want the test to use specific variables, call them out explicitly and explain how you want the agent to use them.

For example, if you associate the test with a DataTable to validate localization, you might include the following explanation in the test prompt: “Use the DataTable variables to validate page headers: validate the login header with {{@login}} and validate the user settings header with {{@user_settings}} to validate user settings page headers.”

Indicate any actions that require manual intervention

The agent has a bias towards trying to accomplish tasks without asking for help. If you know that the test requires a specific unsupported step type, make a note of it in the test prompt.

For example, if you add the following to the prompt - “Ask for help performing the file upload” - the agent knows when to ask for your input.

Mobile tests

For mobile tests, the Test Creation Agent generates a series of tasks for the user to build out. Tasks can include GenAI assertions and relevant flows, but it’s up to the user to test everything out.

To write an effective test prompt for mobile, start with an existing test case or user story and augment it with an explicit description of the particular goals of the test. While it’s not necessary to describe every single click along the way, providing additional high-level details about the full end-to-end user journey can yield more successful results.

For example, you can augment the description “buy Kindle on Amazon” with more details about who the user is and how they should complete the task:
“Verify an existing Prime customer can purchase an item.
Requirements:

• Use search bar to find item
• Item must be a Kindle
• Login using existing Prime account
• Verify checkout flow is successful.”

API tests

For API tests, the Test Creation Agent generates a test outline based on your test prompt and the information you provide about your API.

To write an effective test prompt for API, specify the endpoints you want to test and anything you want to include in the request and response, such as assertions and variables. For example, you could augment the test intent “Verify full CRUD on mabl mailbox endpoints” with the following details:

“Verify interactions with permanent mabl mailboxes:

1. Create: Create a permanent mailbox with name ‘API Test generated mailbox’
2. Read: Get the newly created mailbox and validate its properties are as expected
3. Update: Test updating the mailbox’s metadata, such as name and description
4. Enable/disable: Test enabling or disabling the mailbox works as expected
5. Delete: Clean up at the end of the test by deleting the mailbox”