Evaluation & Chunking Strategy — Databricks Context Associate Practice Question
A representative Databricks Certified Context Engineer Associate (Databricks Context Associate) exam question on Evaluation & Chunking Strategy. Work through it below, then read why each option is right or wrong.
This question is part of the free Databricks Certified Context Engineer Associate practice test.
Short answer
The correct answer is D. Mean Reciprocal Rank (MRR) or Recall@K.
When evaluating chunking strategies for relevance, metrics like Mean Reciprocal Rank (MRR) or Recall@K measure how often the correct answer is found within the top-K retrieved documents. These directly correlate with context quality. Token generation time and cost are performance/metrics of the pipeline but do not measure the *quality* or *relevance* of the retrieved context itself.
The Question
An organization wants to compare two different chunking strategies for their knowledge base before deploying a new RAG pipeline. They have a set of 100 validated user queries with ground truth answers and corresponding retrieved documents. Which metric is most appropriate for empirically determining which strategy yields higher relevance in the retrieved context?
Why D is correct
When evaluating chunking strategies for relevance, metrics like Mean Reciprocal Rank (MRR) or Recall@K measure how often the correct answer is found within the top-K retrieved documents. These directly correlate with context quality. Token generation time and cost are performance/metrics of the pipeline but do not measure the *quality* or *relevance* of the retrieved context itself.
Why the other options are wrong
Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.
Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.
Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.
Key idea: Evaluation & Chunking Strategy
When evaluating chunking strategies for relevance, metrics like Mean Reciprocal Rank (MRR) or Recall@K measure how often the correct answer is found within the top-K retrieved documents. These directly correlate with context quality. Token generation time and cost are performance/metrics of the pipeline but do not measure the *quality* or *relevance* of the retrieved context itself. On the Databricks Context Associate exam, questions in the "Foundations of Context Engineering" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which Databricks Certified Context Engineer Associate domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.
Related Databricks Certified Context Engineer Associate Practice Questions
Official sources used
Verify program and exam details at the source before scheduling.