CoursesFree Quizzes
MicrosoftOptimize generative AI systems and model performance

Cost Optimization & Model Selection — AI-300 Practice Question

A representative Microsoft Machine Learning Operations Engineer Associate (AI-300) exam question on Cost Optimization & Model Selection. Work through it below, then read why each option is right or wrong.

This question is part of the free Microsoft Machine Learning Operations Engineer Associate practice test.

Short answer

The correct answer is D. Implement a routing layer that classifies incoming requests and directs simple tasks to a smaller, faster model while reserving GPT-4 for complex reasoning tasks, reducing average token consumption and per-request cost.

Intelligent routing based on task complexity is a standard cost-optimization pattern: simple tasks consume fewer tokens on cheaper models, complex tasks use GPT-4 only when necessary. This preserves SLA latency by keeping simple requests fast on smaller models. Reducing max_tokens globally risks accuracy loss and may violate SLAs. Wholesale migration to GPT-3.5 sacrifices quality without addressing the core inefficiency. Request queuing with 10-second timeouts violates the <2 second SLA requirement. Azure ML documentation emphasizes model selection and routing as primary levers for cost optimization without quality degradation.

The Question

Your Azure Machine Learning production pipeline processes 100,000 inference requests daily using a GPT-4 deployment. Cost analysis reveals that 40% of requests are simple classification tasks that could be handled by a smaller model. You need to reduce monthly costs by 35% while maintaining SLA latency targets of <2 seconds for 99% of requests. What is the recommended approach?

AMigrate all 100,000 daily requests to GPT-3.5-turbo and accept the 5-10% accuracy decrease as a cost-benefit tradeoff
BImplement request queuing with a 10-second timeout to batch requests together, which will reduce API calls and lower per-request overhead costs
CReduce the max_tokens parameter globally across all requests to 100 tokens to lower token consumption, then increase batch size to 1000 to improve throughput and amortize costs
DImplement a routing layer that classifies incoming requests and directs simple tasks to a smaller, faster model while reserving GPT-4 for complex reasoning tasks, reducing average token consumption and per-request costCorrect

Why D is correct

Intelligent routing based on task complexity is a standard cost-optimization pattern: simple tasks consume fewer tokens on cheaper models, complex tasks use GPT-4 only when necessary. This preserves SLA latency by keeping simple requests fast on smaller models. Reducing max_tokens globally risks accuracy loss and may violate SLAs. Wholesale migration to GPT-3.5 sacrifices quality without addressing the core inefficiency. Request queuing with 10-second timeouts violates the <2 second SLA requirement. Azure ML documentation emphasizes model selection and routing as primary levers for cost optimization without quality degradation.

Why the other options are wrong

Option A: Migrate all 100,000 daily requests to GPT-3.5-turbo and accept the 5-10% accuracy decrease as a cost-benefit tradeoff

Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Option B: Implement request queuing with a 10-second timeout to batch requests together, which will reduce API calls and lower per-request overhead costs

Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Option C: Reduce the max_tokens parameter globally across all requests to 100 tokens to lower token consumption, then increase batch size to 1000 to improve throughput and amortize costs

Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Key idea: Cost Optimization & Model Selection

Intelligent routing based on task complexity is a standard cost-optimization pattern: simple tasks consume fewer tokens on cheaper models, complex tasks use GPT-4 only when necessary. This preserves SLA latency by keeping simple requests fast on smaller models. Reducing max_tokens globally risks accuracy loss and may violate SLAs. Wholesale migration to GPT-3.5 sacrifices quality without addressing the core inefficiency. Request queuing with 10-second timeouts violates the <2 second SLA requirement. Azure ML documentation emphasizes model selection and routing as primary levers for cost optimization without quality degradation. On the AI-300 exam, questions in the "Optimize generative AI systems and model performance" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.

Ready to see how you'd score?

Take the free practice quiz and find out which Microsoft Machine Learning Operations Engineer Associate domains you need to focus on. No signup required.

Take the free quizSee all 220+ practice questions

Practice 5 similar questions

Same cert, same or adjacent domain. Use these after reviewing the explanation.

1Mlflow & Git — AI-300 Practice QuestionDesign and implement an MLOps infrastructure2Model Registry & Versioning — AI-300 Practice QuestionManage the machine learning model lifecycle and operations3Observability & Tracing — AI-300 Practice QuestionDesign and implement a GenAIOps infrastructure4Offline Evaluation & Online Evaluation — AI-300 Practice QuestionImplement quality assurance and observability for generative AI5Azure Devops & Variable Groups — AI-300 Practice QuestionDesign and implement an MLOps infrastructure

Related Microsoft Machine Learning Operations Engineer Associate Practice Questions

Mlflow & Git — AI-300 Practice Question→Model Registry & Versioning — AI-300 Practice Question→Observability & Tracing — AI-300 Practice Question→

Official sources used

Verify program and exam details at the source before scheduling.

learn.microsoft.comConcept Model Management And Deploymentlearn.microsoft.comConcept ML Pipelines