Provider Model Upgrade Cut Question Explained
This worked AI Engineer Interview practice question tests Problem Framing. Try the answer first, then review why C is the best choice.
Short answer
The correct answer is C. Whether the escalation rise is caused by the swap at all, and what evidence would tell you within the rollback window..
The escalation rise is correlated with the swap, not yet attributed to it — and the old model is only available for a limited window, so the framing is 'what evidence, how fast.' Those options both accept the causal link before it's established. The key is to match the exact action in the prompt to the tool, command, or concept that performs that action directly. Here, C. Whether the escalation rise is caused by the swap at all, and what evidence would tell you within the rollback window. is the only option that satisfies the scenario without adding unrelated behavior.
The Question
A provider model upgrade cut latency 25%, but support escalations rose 18% over the following week. Public benchmarks favored the new model; your offline eval set has 40 examples. What is the real decision you have to frame?
Why C is correct
The escalation rise is correlated with the swap, not yet attributed to it — and the old model is only available for a limited window, so the framing is 'what evidence, how fast.' Those options both accept the causal link before it's established. The key is to match the exact action in the prompt to the tool, command, or concept that performs that action directly. Here, C. Whether the escalation rise is caused by the swap at all, and what evidence would tell you within the rollback window. is the only option that satisfies the scenario without adding unrelated behavior.
Why the other options are wrong
Option A does not match the requirement in the prompt. The correct answer is C because The escalation rise is correlated with the swap, not yet attributed to it — and the old model is only available for a limited window, so the framing is 'what evidence, how fast.' Those options both accept the causal link before it's established.
Option B does not match the requirement in the prompt. The correct answer is C because The escalation rise is correlated with the swap, not yet attributed to it — and the old model is only available for a limited window, so the framing is 'what evidence, how fast.' Those options both accept the causal link before it's established.
Option D does not match the requirement in the prompt. The correct answer is C because The escalation rise is correlated with the swap, not yet attributed to it — and the old model is only available for a limited window, so the framing is 'what evidence, how fast.' Those options both accept the causal link before it's established.
Key idea: Problem Framing
Every question is a production situation an AI engineer interviewer can put to you: a failing endpoint, a cost spike, an agent that loops, an eval that disagrees with users. The best answer is almost always the one that measures or contains the problem before it changes the model or the prompt. If you miss a question, note which habit it tests: decompose before you fix, put the security boundary where the side effect happens, and trust an eval only after you check what it covers. The 12 open-ended questions with answer guides are at aijobprep.app/questions/ai-engineer.html.
Ready to test the full skill?
Take the free AI Engineer Interview quiz and find the exact basics to review next. No signup required.
Practice related questions
Same skill area or adjacent concept. Use these after reviewing the explanation.