Skip to main content

Test sets

Modified: 6 Nov 2026 Trimble Agent Studio Workflows Help

A test set is a reusable collection of test cases (user prompts with optional expected outputs) that you can run against agents in evaluation jobs.

You can access them by navigating to Evaluate > Test Sets.

The Test Sets page displays all test sets that you have access to.

Figure 1. Test sets

Test sets catalog

Use Search, Filter, and Sort in the toolbar to narrow the catalog.

Switch between grid and list layouts with the layout toggle.

To create a new test set, select New. Give it a name and a description, then attach the initial cases as a JSON file. Select Download template for the format. Create stays disabled until a file is attached, so prepare at least one case first. You can also draft a set from the New Evaluation wizard with Generate with AI. For adding cases later, see Add test cases.

What makes a good set

  • Build it from real questions, not from questions the agent is already good at.
  • Include awkward cases: ambiguous questions, questions the agent should refuse, and questions that need the knowledge library rather than general knowledge.
  • Keep the set stable. Cases carry a version number, so grow the set by adding cases rather than rewriting it.

Test set details

Open a test set to manage its cases and sharing. Use Download All to export cases, select checkboxes to delete cases in bulk, and open the Sharing tab so the person who curates questions can differ from the person who runs evaluations.

Submit

Table of Contents