RAG & Vector Search
A retail company wants a customer-support assistant on Databricks that answers questions about its internal return policies, which change weekly. Fine-tuning a model on each policy update is considered too slow and costly. Latency must stay low and answers must cite current policy text. Which application pattern best fits these requirements?
Retrieval-augmented generation (RAG) separates knowledge from the model: policy documents are chunked, embedded, and indexed in Mosaic AI Vector Search, so weekly changes only require re-indexing, not retraining. Retrieved passages also provide citable grounding. Continued pre-training every week is exactly the slow, costly retraining cycle the company rejected, and it cannot guarantee citations. Static few-shot prompts freeze knowledge at prompt-authoring time, so weekly policy changes would make answers stale, and prompt size limits how much policy text fits. A SQL-writing agent suits structured analytics, but return policies are unstructured prose, and text-to-SQL adds latency and failure modes without solving grounding on documents.