AnthropicEvaluation, Testing, and Optimization

Evaluation & Graders — CCAR-P Practice Question

A representative Claude Certified Architect - Professional (CCAR-P) exam question on Evaluation & Graders. Work through it below, then read why each option is right or wrong.

This question is part of the free Claude Certified Architect - Professional practice test.

Short answer

The correct answer is D. A mixed evaluation that also checks source validity, claim support, coverage, tool behavior, and expert review for disputed cases.

One style-oriented judge cannot measure factual grounding or process correctness. Combining deterministic checks, task-specific judges, and expert review gives broader evidence.

The Question

A research agent is evaluated only by an LLM judge that scores writing quality. What is missing?

AA longer answer limit

BA second identical writing judge

CMore decorative formatting

DA mixed evaluation that also checks source validity, claim support, coverage, tool behavior, and expert review for disputed casesCorrect

Why D is correct

One style-oriented judge cannot measure factual grounding or process correctness. Combining deterministic checks, task-specific judges, and expert review gives broader evidence.

Why the other options are wrong

Option A: A longer answer limit

Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Option B: A second identical writing judge

Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Option C: More decorative formatting

Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Key idea: Evaluation & Graders

One style-oriented judge cannot measure factual grounding or process correctness. Combining deterministic checks, task-specific judges, and expert review gives broader evidence. On the CCAR-P exam, questions in the "Evaluation, Testing, and Optimization" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.

Ready to see how you'd score?

Take the free practice quiz and find out which Claude Certified Architect - Professional domains you need to focus on. No signup required.

Take the free quiz See all 20+ practice questions

Practice 5 similar questions

Same cert, same or adjacent domain. Use these after reviewing the explanation.

Take the quiz →

1Discovery & Requirements — CCAR-P Practice QuestionSolution Design and Architecture 2Model Routing & Governance — CCAR-P Practice QuestionClaude Models, Prompting, and Context Engineering 3RAG & Evaluation — CCAR-P Practice QuestionIntegration 4Evals & Release Gate — CCAR-P Practice QuestionEvaluation, Testing, and Optimization 5Governance & Data Retention — CCAR-P Practice QuestionGovernance, Safety, and Risk Management

Related Claude Certified Architect - Professional Practice Questions

Discovery & Requirements — CCAR-P Practice Question→Model Routing & Governance — CCAR-P Practice Question→RAG & Evaluation — CCAR-P Practice Question→

Official sources used

Verify program and exam details at the source before scheduling.

AnthropicDemystifying Evals For AI Agents