Quantization & Memory Optimization — NCP-GENL Practice Question
A representative NVIDIA-Certified Professional Generative AI LLMs (NCP-GENL) exam question on Quantization & Memory Optimization. Work through it below, then read why each option is right or wrong.
This question is part of the free NVIDIA-Certified Professional Generative AI LLMs practice test.
Short answer
The correct answer is B. Apply INT8 quantization to reduce model weights and activations, which decreases memory footprint and enables larger batches while improving compute throughput.
Quantization directly reduces model size and activation memory, enabling larger batch sizes which improves throughput and amortizes latency overhead. Increasing GPUs adds communication latency and cost without addressing the root memory constraint. Reducing sequence length is a functional limitation, not an optimization. Gradient checkpointing is a training technique and provides no benefit during inference.
The Question
You are deploying a 70B parameter LLM in production using TensorRT-LLM and observe that GPU memory utilization reaches 80% during inference, limiting batch size to 4. Your latency requirement is 150ms per token, and you currently achieve 180ms. Which optimization strategy should you prioritize first to address both memory and latency constraints simultaneously?
Why B is correct
Quantization directly reduces model size and activation memory, enabling larger batch sizes which improves throughput and amortizes latency overhead. Increasing GPUs adds communication latency and cost without addressing the root memory constraint. Reducing sequence length is a functional limitation, not an optimization. Gradient checkpointing is a training technique and provides no benefit during inference.
Why the other options are wrong
Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Option D does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Key idea: Quantization & Memory Optimization
Quantization directly reduces model size and activation memory, enabling larger batch sizes which improves throughput and amortizes latency overhead. Increasing GPUs adds communication latency and cost without addressing the root memory constraint. Reducing sequence length is a functional limitation, not an optimization. Gradient checkpointing is a training technique and provides no benefit during inference. On the NCP-GENL exam, questions in the "Model Optimization" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which NVIDIA-Certified Professional Generative AI LLMs domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.
Related NVIDIA-Certified Professional Generative AI LLMs Practice Questions
Official sources used
Verify program and exam details at the source before scheduling.