Evaluation & Rubric — CCAO-F Practice Question
A representative Claude Certified Associate - Foundations (CCAO-F) exam question on Evaluation & Rubric. Work through it below, then read why each option is right or wrong.
This question is part of the free Claude Certified Associate - Foundations practice test.
Short answer
The correct answer is B. Define a rubric for factual accuracy, policy compliance, clarity, and actionability, then score a representative set blind.
A task-specific rubric and representative blind sample make judgments repeatable and expose tradeoffs. Length, model preference, and easy-only examples are weak proxies.
The Question
A team compares two prompt versions for customer-email drafts. Reviewers keep disagreeing about which is better. What is the best improvement to the evaluation?
Why B is correct
A task-specific rubric and representative blind sample make judgments repeatable and expose tradeoffs. Length, model preference, and easy-only examples are weak proxies.
Why the other options are wrong
Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Option D does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Key idea: Evaluation & Rubric
A task-specific rubric and representative blind sample make judgments repeatable and expose tradeoffs. Length, model preference, and easy-only examples are weak proxies. On the CCAO-F exam, questions in the "Output Evaluation and Validation" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which Claude Certified Associate - Foundations domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.
Related Claude Certified Associate - Foundations Practice Questions
Official sources used
Verify program and exam details at the source before scheduling.