Tech SkillsSystem Reasoning

Operations Agent 28 Tools Question Explained

This worked AI Engineer Interview practice question tests System Reasoning. Try the answer first, then review why A is the best choice.

Short answer

The correct answer is A. Which tool pairs get confused, and whether errors are routing mistakes or execution failures — retries only address the latter..

Routing errors and execution errors have different fixes — a router exposing a relevant subset versus retries on transient failures — so the confusion matrix decides the architecture. The key is to match the exact action in the prompt to the tool, command, or concept that performs that action directly. Here, A. Which tool pairs get confused, and whether errors are routing mistakes or execution failures — retries only address the latter. is the only option that satisfies the scenario without adding unrelated behavior.

The Question

An operations agent has 28 tools and picks the wrong one often enough to need manual cleanup. The platform team proposes longer tool descriptions plus five retries. What do you measure before accepting any fix?

AWhich tool pairs get confused, and whether errors are routing mistakes or execution failures — retries only address the latter.Correct
BWhether a bigger model selects tools more accurately with the same descriptions.
CAverage task completion rate before and after the description change.
DToken cost of the longer descriptions across a week of traffic.

Why A is correct

Routing errors and execution errors have different fixes — a router exposing a relevant subset versus retries on transient failures — so the confusion matrix decides the architecture. The key is to match the exact action in the prompt to the tool, command, or concept that performs that action directly. Here, A. Which tool pairs get confused, and whether errors are routing mistakes or execution failures — retries only address the latter. is the only option that satisfies the scenario without adding unrelated behavior.

Why the other options are wrong

Option B: Whether a bigger model selects tools more accurately with the same descriptions.

Option B does not match the requirement in the prompt. The correct answer is A because Routing errors and execution errors have different fixes — a router exposing a relevant subset versus retries on transient failures — so the confusion matrix decides the architecture.

Option C: Average task completion rate before and after the description change.

Option C does not match the requirement in the prompt. The correct answer is A because Routing errors and execution errors have different fixes — a router exposing a relevant subset versus retries on transient failures — so the confusion matrix decides the architecture.

Option D: Token cost of the longer descriptions across a week of traffic.

Option D does not match the requirement in the prompt. The correct answer is A because Routing errors and execution errors have different fixes — a router exposing a relevant subset versus retries on transient failures — so the confusion matrix decides the architecture.

Key idea: System Reasoning

Every question is a production situation an AI engineer interviewer can put to you: a failing endpoint, a cost spike, an agent that loops, an eval that disagrees with users. The best answer is almost always the one that measures or contains the problem before it changes the model or the prompt. If you miss a question, note which habit it tests: decompose before you fix, put the security boundary where the side effect happens, and trust an eval only after you check what it covers. The 12 open-ended questions with answer guides are at aijobprep.app/questions/ai-engineer.html.

Ready to test the full skill?

Take the free AI Engineer Interview quiz and find the exact basics to review next. No signup required.

Practice related questions

Same skill area or adjacent concept. Use these after reviewing the explanation.

Related AI Engineer Interview Practice Questions