Explanation:
Semantic cache metric: Candidate-set recall and precision; Integrated system check: An end-to-end evaluation against the curated dataset; Regression dataset: Add cases for the new capability to the curated dataset.
The semantic cache is acting as a retrieval/shortlisting component, so recall and precision are the appropriate measures of whether expected agent candidates are surfaced and whether irrelevant candidates are avoided.
The pull request also changes the selector prompt and adds an agent, so component-only tests are insufficient; an end-to-end evaluation must exercise retrieval, routing, the new agent, and final response generation together. Finally, the version-controlled regression dataset should evolve with the capability. Adding representative cases for the new agent keeps expected routing and responses aligned with the code and prompt changes in the pull request. Microsoft Foundry evaluation guidance treats curated datasets as reusable regression assets for CI/CD quality gates. Together, these three selections test retrieval quality, integrated behavior, and regression coverage at the correct layers. The evaluation should also preserve correlation identifiers and version information where possible so a failed score can be traced back to the exact agent, model, tool call, or retrieval step that produced it. This turns the metric into an actionable diagnostic rather than only a dashboard number.
Official Microsoft reference: Microsoft Foundry - evaluation datasets and CI/CD evaluation