RAG Implementation & Prompt Engineering — AI-103 Practice Question
A representative Microsoft Azure AI Apps and Agents Developer Associate (AI-103) exam question on RAG Implementation & Prompt Engineering. Work through it below, then read why each option is right or wrong.
This question is part of the free Microsoft Azure AI Apps and Agents Developer Associate practice test.
Short answer
The correct answer is A. Add a system prompt instruction that explicitly tells the model to only answer based on retrieved documents and to refuse if the answer is not in the context.
Prompt engineering is the most direct control for enforcing that the model respects retrieval boundaries. An explicit system prompt instruction that requires the model to stay within retrieved context and refuse out-of-scope questions directly addresses hallucination. Retrieving more documents increases context volume but does not prevent hallucination if the prompt does not enforce grounding. Tightening the retrieval threshold may improve quality but does not instruct the model on how to use the context. Model version upgrades improve reasoning but do not guarantee grounding without explicit prompt guidance. The system prompt is the primary, most direct lever.
The Question
Your organization is implementing a RAG system for an internal knowledge base. You have indexed 10,000 documents in Azure AI Search. When a user asks a question, the system retrieves the top 5 documents and sends them to Azure OpenAI. However, the model sometimes generates responses that are not supported by any of the retrieved documents. What is the most direct way to reduce this behavior?
Why A is correct
Prompt engineering is the most direct control for enforcing that the model respects retrieval boundaries. An explicit system prompt instruction that requires the model to stay within retrieved context and refuse out-of-scope questions directly addresses hallucination. Retrieving more documents increases context volume but does not prevent hallucination if the prompt does not enforce grounding. Tightening the retrieval threshold may improve quality but does not instruct the model on how to use the context. Model version upgrades improve reasoning but do not guarantee grounding without explicit prompt guidance. The system prompt is the primary, most direct lever.
Why the other options are wrong
Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (A) is the only one that fully meets every constraint stated in the question.
Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (A) is the only one that fully meets every constraint stated in the question.
Option D does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (A) is the only one that fully meets every constraint stated in the question.
Key idea: RAG Implementation & Prompt Engineering
Prompt engineering is the most direct control for enforcing that the model respects retrieval boundaries. An explicit system prompt instruction that requires the model to stay within retrieved context and refuse out-of-scope questions directly addresses hallucination. Retrieving more documents increases context volume but does not prevent hallucination if the prompt does not enforce grounding. Tightening the retrieval threshold may improve quality but does not instruct the model on how to use the context. Model version upgrades improve reasoning but do not guarantee grounding without explicit prompt guidance. The system prompt is the primary, most direct lever. On the AI-103 exam, questions in the "Implement generative AI and agentic solutions" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which Microsoft Azure AI Apps and Agents Developer Associate domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.
Related Microsoft Azure AI Apps and Agents Developer Associate Practice Questions
Official sources used
Verify program and exam details at the source before scheduling.