Evals & Release Gate — CCAR-P Practice Question
A representative Claude Certified Architect - Professional (CCAR-P) exam question on Evals & Release Gate. Work through it below, then read why each option is right or wrong.
Short answer
The correct answer is A. Not until the risk-specific release threshold is met and the violation regression is understood.
Averages must not hide critical regressions. Risk-weighted gates should block a release that worsens an explicitly protected safety outcome.
The Question
An assistant's overall success rate rises from 84% to 89%, but policy-violation failures double from 0.2% to 0.4%. Should the release proceed?
Why A is correct
Averages must not hide critical regressions. Risk-weighted gates should block a release that worsens an explicitly protected safety outcome.
Why the other options are wrong
Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (A) is the only one that fully meets every constraint stated in the question.
Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (A) is the only one that fully meets every constraint stated in the question.
Option D does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (A) is the only one that fully meets every constraint stated in the question.
Key idea: Evals & Release Gate
Averages must not hide critical regressions. Risk-weighted gates should block a release that worsens an explicitly protected safety outcome. On the CCAR-P exam, questions in the "Evaluation, Testing, and Optimization" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which Claude Certified Architect - Professional domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.