Download assertions

To help you cover end-to-end testing scenarios, the mabl Trainer detects and automatically adds assertions for file downloads from your web app. By default, the Trainer asserts against the name of the downloaded file, but you can also edit the assertion to validate the file contents or other metadata.

This article explains how to work with download assertions in the Trainer, including how to…

• Assert on file metadata
• Assert on file contents
• Replay download assertions

Assert on file metadata

To assert on file metadata, including filename, file type, and file size, click on Edit assertions. Alternatively, click on the edit pencil for the download assertion step.

Dynamically generated filenames

If your application creates a dynamically generated filename for the file download, you may try one of the following workflows to assert that the file has downloaded:

• Use partial string matching - if a portion of the filename is predictable, use a partial match assertion type, such as contains, starts with, ends with or matches regex.
• Use mabl variable syntax - if you know how the filename is generated, or if you can extract the filename from the page before downloading, store the filename in a variable and assert against that filename using mabl variable syntax, for example {{@filename_var}}.
• Remove the filename assertion - if you do not know what the filename will be before downloading it, remove the filename from the download assertion and assert against other attributes, such as the MIME type or file size.

Assert on file contents

You can also validate the content of downloaded files with visual assertions. The following file types are supported for file content assertions. Note that file metadata assertions (filename, file type, and file size) are supported for all file types.

*Excel files are converted to CSV for analysis.

To assert on the contents of a downloaded file, take the following steps:

1. Edit the download assertion step.
2. Provide a natural language prompt that describes what you would like to validate.
3. If you wish to review file contents, click on View file.
4. When you are ready to test your assertion, click on Test assertion.

Assertion criteria

The first time you test the assertion, mabl generates assertion criteria. These criteria are sent with the assertion description to the LLM to provide a consistent set of criteria to evaluate as part of your assertion.

If the criteria don’t match your intent, add, edit, or delete the criteria as needed. To replace all criteria, click on the Regenerate criteria button in the Test assertion dropdown.

Screenshots and viewports

When asserting against a file download, visual assertions include both the contents of the downloaded file and a screenshot of the page it was downloaded from. With the page screenshot, you can create validations on data across the file and the app. For example, you could assert that the file data from a spreadsheet aligns with the product information that is visible on the current page.

Keep in mind that the screenshot only includes the portion of the page that is currently visible in the viewport. If you assert on something that isn’t fully visible in the viewport, any parts of the page that are off-screen will not be included in the evaluation. To resize the window, add a “set viewport” step before the assertion: + (Add step) > Set viewport.

Running tests with visual assertions

Visual download assertions are supported in Trainer replay, local, and cloud execution. When triggering a local run from the mabl CLI, you must use the flag --allow-billable-features to enable visual assertions. For example:

mabl tests run --id {test-id} --allow-billable-features

Replay download assertions

To replay a download assertion in the Trainer, move the Trainer cursor to just above the click step that triggers the download and click play. The Trainer automatically elides numbers that are added to repeatedly downloaded files so that foo (1).zip becomes foo.zip for the assertion comparison. This feature enables replay testing without requiring manual file cleanup.

Due to Chrome limits, you may only download a file 100 times. If you replay a download step in the mabl Trainer more than 100 times, Chrome automatically opens a save file modal for every additional download, preventing further replay of the download step. To restore replay functionality, remove the downloaded files from your machine.

Limitations

Note the following limitations when downloading files in your test:

• Download assertions are supported for recorded click steps. Click steps that use CSS or XPath do not trigger a download assertion.
• The Trainer cannot validate multiple downloads in parallel, either for file metadata or file content.
• mabl automatically cleans up downloaded files at the end of each training session and test execution. For this reason, it is not possible to manually inspect the content of downloaded files outside of a training session.
• If a download assertion fails, the test stops running and fails. Other failure behaviors, such as “fail at end of test” are not currently supported for download assertions.

File metadata limitations

• Download assertions do not currently support the use of wildcards (*) within filenames.
• Download assertions aren’t supported for filenames that contain accented characters.

File size limitations

• The size limit for downloaded file varies by type:
  • Text, CSV, Excel: 7MB
  • Images: 30MB
  • Everything else (audio, video, docs/PDFs): 50MB
• Spreadsheets are limited to 10 sheets and 1,000 rows per sheet.