A Genie space author wants to run a structured test to measure whether Genie correctly answers a predefined set of business questions. The author also wants to compare Genie's responses against expected SQL results for scoring. Which Genie feature is purpose-built for this use case?
Answer & explanation
Correct answer: E
WHY E: The documentation describes benchmarks as the purpose-built feature for structured evaluation. Authors create a collection of test questions, optionally with an expected SQL answer for each; when the benchmark runs, Genie's responses are compared against the expected results and scored. Benchmarks evaluate the space: they are not added to Genie's context and do not influence how Genie generates future responses, unlike example SQL queries, which do.
WHY NOT A: The Monitoring tab shows live production questions and user feedback for qualitative review. It does not compare responses against expected SQL outputs or produce a structured, scored accuracy report.
WHY NOT B: Thumbs-up and thumbs-down feedback gives authors a qualitative signal for spotting problem areas; it is not a structured scoring mechanism that compares responses with expected SQL.
WHY NOT C: Inspect mode is a Beta feature that improves the accuracy of an individual query by having Genie review its own SQL. It is not a batch evaluation tool that compares responses against a predefined expected answer set.
WHY NOT D: The general instructions text block is part of the context Genie uses to generate responses; Databricks does not run it as a regression test suite. Putting expected outputs there would instruct Genie to match them instead of evaluating against them independently.