Agentic test authoring for web apps

Creating and maintaining reliable browser tests can take up a lot of time. With the mabl agent, you can streamline this process by describing what you want the test to accomplish in natural language. The agent then takes on the work of generating test steps, basing its decisions on contextual information from your workspace, so that you can focus on more strategic work.

This article explains how to use the agentic test authoring for browser tests:

• Create a new browser test with the agent
• Manage agent execution
• Generate steps in existing tests

Create a new browser test with the agent

To create a new browser test with the mabl agent, go to New test > Browser test in the app and provide your test prompt details. If you aren’t sure what information to provide, see our article on writing an effective test prompt.

Optionally, you may also specify up to three tests for the agent to reference for screenshots of the relevant user flow.

When you submit a prompt, the agent searches your mabl workspace for additional context, including an application summary and related tests, flows, and snippets. Using this context, the agent builds a test outline. If the outline doesn’t look correct, send followup messages to refine it until it aligns with your goals.

Test references

When planning the new test, the agent pulls screenshots from your relevant test runs to serve as context. You can either:

• Provide test references when you submit your initial prompt, OR
• Let the agent automatically search your workspace and identify up to 3 relevant test references for you.

To review and manage test references, expand the test references icon. If needed, you can manually remove or add test references to ensure the outline aligns with your objectives.

Test information

The agent generates high-level details for your test and displays them in the Test information section, including:

• Test name
• Test case
• Test description
• Application
• Environment
• URL
• Credentials

If any of the information is incorrect, send a followup message with your corrections. For example, “In test information, change the Application to ‘Customer Portal’ and update the credentials to ‘Sample customer’.”

Test outline

In the Test outline section, the agent generates a detailed approach to implementing the test, including:

• Implementation notes - the general idea of the test
• Tasks - each task includes a suggested validation for the end of the task
• Suggested flows and snippets - each flow/snippet includes a reasoning for why it was suggested and a confidence level: “preferred” indicates the agent is more likely to use the flow/snippet, “secondary” indicates the agent is saving the flow/snippet as an alternative option.
• Screenshots - if there are test references, the agent uses screenshots from their test runs as context when generating the outline.

To refine the test outline, send followup messages with your specific corrections. For example, “In the test outline, remove the flow ‘Admin Login Flow’ from the first task.” If the screenshots don’t look correct, double check the test references and manually update them as needed.

Create the test

When you are happy with the outline, click on Next and add in any missing details on the test creation form, such as credentials or DataTables. Then select a build approach:

• Cloud - the agent builds out your test in the cloud. Cloud-based authoring supports up to 10 sessions in parallel. Learn more.
• Local - the agent builds out your test locally in the mabl Trainer.

Then click on Create test to start building the test.

Monitor agent execution

The agent uses the following context to generate test steps:

• A screenshot of the app
• The task plan
• Any existing test steps located before the cursor
• The page’s HTML/DOM structure

Steps are bundled together into step groups based on the test outline.

To monitor agent execution in the cloud, see the article on monitoring cloud-based authoring sessions.

Understanding agent activity

For each step, the agent follows a cycle:

• Plan - plans the next action based on the prompt and screenshots
• Act - visually locates and interacts with elements on the screen
• Reflect - reflects on what changed to determine whether the action succeeded.

In local Trainer session, you can click on the clock icon to view agent activity and review decisions.

Manual intervention

When the agent builds the test locally, avoid interacting with the browser window to prevent the agent from getting into an unexpected state.

If the agent appears to be getting off track, pause it and try one of the following manual interventions:

• Edit the task plan - if the agent appears stuck or is repeating actions, it may be because the expected change wasn’t detected. In this case, pause the agent and refine the task plan.
• Train steps manually - in ambiguous situations, the agent may get off track if it doesn’t know enough about your app to make good decisions. If this happens, switch to manual mode to add in tasks and steps manually.

Agentic test authoring supports most common UI interactions, including filling out forms, selecting from date pickers and dropdowns, and clicking on elements. If the task plan includes unsupported interactions, the agent may prompt you to take over training.

Restarting the agent

When you restart the agent, it updates the plan based on the edited prompt and any existing steps located before the cursor. It has no awareness of steps located after the cursor.

Generate steps in existing tests

If you need to add or retrain a set of steps, you can use the Generate button to build out steps based on a natural language prompt. This functionality is available in both new and existing browser tests:

Before you generate new steps in the Trainer, make sure your app is in the correct state. The Trainer cursor must be located in the position where you want to add in new steps. If you are regenerating steps for an existing task, make sure the cursor is located just below the task description step.

Click on the Generate button to access the prompt modal. Then describe what your test needs and click on Generate steps. The mabl agent will use this prompt to generate one or more steps.