CoursesFree Quizzes
Google CloudServing and scaling models

Vertex AI & Online Inference — PMLE Practice Question

A representative Google Cloud Professional Machine Learning Engineer (PMLE) exam question on Vertex AI & Online Inference. Work through it below, then read why each option is right or wrong.

This question is part of the free Google Cloud Professional Machine Learning Engineer practice test.

Short answer

The correct answer is C. Configure the endpoint to pre-load the model into memory during replica initialization and set a higher minimum replica count.

Cold-start latency is caused by model loading when a new replica is initialized. Pre-loading the model into memory during initialization ensures it is ready before serving traffic, and maintaining a higher minimum replica count reduces the frequency of new replica creation. Batch prediction is asynchronous and unsuitable for real-time latency requirements. Request batching does not address cold starts; it increases throughput but not latency for individual requests. GPU acceleration speeds computation but not model loading itself.

The Question

You are deploying a large language model for real-time inference on Vertex AI. Your application requires sub-500ms latency for 99th percentile requests, but you observe that cold starts and model loading are causing tail latencies to exceed 2 seconds. You have already optimized the model weights and quantized the model. Which configuration change will most directly address the cold-start latency issue?

ADeploy the model using Vertex AI Batch Prediction instead of an online endpoint to reduce initialization overhead
BEnable GPU acceleration and use a larger machine type to speed up model loading and inference
CConfigure the endpoint to pre-load the model into memory during replica initialization and set a higher minimum replica countCorrect
DIncrease the minimum number of replicas for the endpoint and enable request batching to amortize model loading overhead

Why C is correct

Cold-start latency is caused by model loading when a new replica is initialized. Pre-loading the model into memory during initialization ensures it is ready before serving traffic, and maintaining a higher minimum replica count reduces the frequency of new replica creation. Batch prediction is asynchronous and unsuitable for real-time latency requirements. Request batching does not address cold starts; it increases throughput but not latency for individual requests. GPU acceleration speeds computation but not model loading itself.

Why the other options are wrong

Option A: Deploy the model using Vertex AI Batch Prediction instead of an online endpoint to reduce initialization overhead

Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (C) is the only one that fully meets every constraint stated in the question.

Option B: Enable GPU acceleration and use a larger machine type to speed up model loading and inference

Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (C) is the only one that fully meets every constraint stated in the question.

Option D: Increase the minimum number of replicas for the endpoint and enable request batching to amortize model loading overhead

Option D does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (C) is the only one that fully meets every constraint stated in the question.

Key idea: Vertex AI & Online Inference

Cold-start latency is caused by model loading when a new replica is initialized. Pre-loading the model into memory during initialization ensures it is ready before serving traffic, and maintaining a higher minimum replica count reduces the frequency of new replica creation. Batch prediction is asynchronous and unsuitable for real-time latency requirements. Request batching does not address cold starts; it increases throughput but not latency for individual requests. GPU acceleration speeds computation but not model loading itself. On the PMLE exam, questions in the "Serving and scaling models" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.

Ready to see how you'd score?

Take the free practice quiz and find out which Google Cloud Professional Machine Learning Engineer domains you need to focus on. No signup required.

Take the free quizSee all 207+ practice questions

Practice 5 similar questions

Same cert, same or adjacent domain. Use these after reviewing the explanation.

1Automl & Vertex AI — PMLE Practice QuestionArchitecting low-code AI solutions2Experiment Tracking & ML Metadata — PMLE Practice QuestionCollaborating to manage data and models3Hyperparameter Tuning & Foundation Models — PMLE Practice QuestionScaling prototypes into ML models4Feature Engineering & Transformation Consistency — PMLE Practice QuestionAutomating and orchestrating ML pipelines5Responsible AI & Bias Detection — PMLE Practice QuestionMonitoring AI solutions

Related Google Cloud Professional Machine Learning Engineer Practice Questions

Automl & Vertex AI — PMLE Practice Question→Experiment Tracking & ML Metadata — PMLE Practice Question→Hyperparameter Tuning & Foundation Models — PMLE Practice Question→

Official sources used

Verify program and exam details at the source before scheduling.

cloud.google.comPredictions: Overviewcloud.google.comIntroduction Unified Platform