A development team is building a conversational application with Snowflake Cortex Analyst to allow business users to ask follow-up questions about structured dat a. They are specifically designing the multi-turn conversation support and considering the underlying LLM choices for components like the summarization agent. Which of the following statements accurately reflects how Cortex Analyst handles conversational context and best practices for selecting an LLM for its summarization agent?
Correct Answer: B
Option B is correct. Cortex Analyst introduces an additional LLM summarization agent before the original workflow to handle multi-turn conversations. This agent distills the conversation history, and Llama 3.1 70B was found to be highly effective for this task with a low error rate (96.5% rated as good), even considering the latency-performance tradeoff. Option A is incorrect. Directly passing the entire raw conversation history to every LLM call can lead to longer inference times, more non-determinism, and degraded overall performance due to multitasking. Option C is incorrect. While previous context is used, Cortex Analyst's multi-turn support involves an LLM summarization agent to rewrite the current question based on conversation history, not just reuse cached SQL query results. Option D is incorrect. While there is a latency- performance tradeoff, Llama 3.1 8B showed an approximate 5% error rate in conversation history rewriting, making Llama 3.1 70B the preferred choice for its higher accuracy despite being a larger model. The goal is to select the smallest model that 'satisfies the need', which in this case prioritizes summarization quality. Option E is incorrect. Summarization is a natural language task (rewriting the question based on context), distinct from SQL query generation, which is handled by other agents in the Cortex Analyst workflow.