Comprehensive and Detailed Explanation From Microsoft AB-620 Study Guide: A predefined test set with expected outcomes is the only option that satisfies consistency, grounding comparison, and repeatable regression analysis. The fixed prompts ensure that each version receives the same inputs. Verified expected responses provide the baseline against which meaning, quality, keywords, or exact output can be scored.
Rerunning the same set produces comparable results over time, allowing the team to calculate whether a knowledge-base change improved or degraded performance. The manual test pane is valuable for interactive debugging and inspecting the activity map, but it does not provide the same controlled batch or historical comparison. Channel usage metrics describe adoption and operational behavior, not response correctness.
Transcript keyword searches can reveal patterns after deployment but do not establish a golden answer or a controlled experiment. The evaluation method should match the content: compare-meaning for legitimate paraphrases, exact match for fixed values, keyword match for required terms, or a custom rubric for domain- specific quality. Results should be retained or exported when a longer audit history is required. Study Guide alignment: Test and manage agents > Evaluate agent performance > Create a test set; Choose an evaluation method; Review test results.