Upon completing the development of a RAG-powered question-answering system, a team performs a crucial step prior to its release. They execute mlflow.genai.evaluate() on a predefined test set of 200 query-response examples, employing LLM-based evaluators to gauge the system's veracity, contextual pertinence, and factual basis. Which stage of the Generative AI application lifecycle does this action most accurately represent?
Answer & explanation
Correct answer: A
WHY A is correct: Offline evaluation (pre-deployment evaluation) involves running the model against a fixed, curated labeled dataset before release to quantify quality. mlflow.genai.evaluate() on a benchmark set with LLM judges measuring faithfulness, relevance, and groundedness is the canonical offline evaluation workflow for Gen AI applications.
WHY NOT C: Online monitoring occurs after deployment on real production traffic. The distinguishing factor is the data source: curated static benchmark (offline evaluation) versus live user traffic (online monitoring). Using LLM judges is common to both phases.
WHY NOT B: Data preparation involves collecting, chunking, and indexing source documents. While evaluation results may inform iterative data preparation improvements, the activity of running mlflow.genai.evaluate() on a labeled test set is evaluation, not data preparation.
WHY NOT D: mlflow.genai.evaluate() is an evaluation and metric-logging function that records to an MLflow experiment run. Model registration uses mlflow.register_model() to package a model artifact in Unity Catalog — a distinct operation.