Bedrock & Model Distillation — AWS GenAI Dev Practice Question
A representative AWS Generative AI Developer (AWS GenAI Dev) exam question on Bedrock & Model Distillation. Work through it below, then read why each option is right or wrong.
Short answer
The correct answer is A. Use Bedrock model distillation with Claude 3.5 Sonnet as the teacher model to create a distilled student model based on a smaller foundation model, then deploy the student model for production inference.
Model distillation in Bedrock uses a large, capable teacher model (Claude 3.5 Sonnet) to train a smaller, cheaper student model that mimics the teacher's behavior on the specific task. With 3 months of high-quality production data and well-defined categories, the distilled model can learn the teacher's response patterns for this narrow domain. The student model runs on a smaller foundation model with significantly lower per-token inference costs, achieving the 60%+ cost reduction. Distillation is ideal when you have abundant labeled data from a high-quality model and the task is well-scoped.
The Question
A company runs a customer support chatbot on Amazon Bedrock using Claude 3.5 Sonnet. The chatbot handles 500,000 requests per day with consistent, high-quality responses. However, the monthly inference cost has reached $45,000 and the CTO wants to reduce it by at least 60% without degrading response quality. The team has 3 months of production request-response pairs logged in S3. The responses follow predictable patterns across 15 well-defined support categories. Which approach best meets the cost reduction target while maintaining response quality?
Why A is correct
Model distillation in Bedrock uses a large, capable teacher model (Claude 3.5 Sonnet) to train a smaller, cheaper student model that mimics the teacher's behavior on the specific task. With 3 months of high-quality production data and well-defined categories, the distilled model can learn the teacher's response patterns for this narrow domain. The student model runs on a smaller foundation model with significantly lower per-token inference costs, achieving the 60%+ cost reduction. Distillation is ideal when you have abundant labeled data from a high-quality model and the task is well-scoped.
Why the other options are wrong
Fine-tuning Claude 3.5 Sonnet creates a customized version of the same large model. While it might improve task performance, the per-token inference cost remains tied to the Sonnet pricing tier. Fine-tuning does not reduce the model size or fundamentally change the cost structure, so it cannot achieve a 60% cost reduction.
Adding few-shot examples to every request actually increases input token count and cost per request. While Haiku is cheaper, stuffing prompts with examples inflates token usage, partially negating the savings. This approach also increases latency and is fragile — prompt engineering alone cannot reliably replicate the quality of a model specifically trained on the task distribution.
Provisioned throughput provides predictable pricing and guaranteed capacity but does not offer 60% cost savings over on-demand pricing for the same model. Response caching helps for exact duplicates but customer support queries vary in wording even within the same category, resulting in low cache hit rates. This approach optimizes infrastructure, not the fundamental cost-per-inference.
Key idea: Bedrock & Model Distillation
Why A is correct: Model distillation in Bedrock uses a large, capable teacher model (Claude 3.5 Sonnet) to train a smaller, cheaper student model that mimics the teacher's behavior on the specific task. With 3 months of high-quality production data and well-defined categories, the distilled model can learn the teacher's response patterns for this narrow domain. The student model runs on a smaller foundation model with significantly lower per-token inference costs, achieving the 60%+ cost reduction. Distillation is ideal when you have abundant labeled data from a high-quality model and the task is well-scoped. Why B is wrong: Fine-tuning Claude 3.5 Sonnet creates a customized version of the same large model. While it might improve task performance, the per-token inference cost remains tied to the Sonnet pricing tier. Fine-tuning does not reduce the model size or fundamentally change the cost structure, so it cannot achieve a 60% cost reduction. Why C is wrong: Adding few-shot examples to every request actually increases input token count and cost per request. While Haiku is cheaper, stuffing prompts with examples inflates token usage, partially negating the savings. This approach also increases latency and is fragile — prompt engineering alone cannot reliably replicate the quality of a model specifically trained on the task distribution. Why D is wrong: Provisioned throughput provides predictable pricing and guaranteed capacity but does not offer 60% cost savings over on-demand pricing for the same model. Response caching helps for exact duplicates but customer support queries vary in wording even within the same category, resulting in low cache hit rates. This approach optimizes infrastructure, not the fundamental cost-per-inference. On the AWS GenAI Dev exam, questions in the "Optimizing Foundation Models" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which AWS Generative AI Developer - Professional domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.