Free Quizzes
NVIDIAExperimentation and Evaluation

Evaluation Metrics & Perplexity — NCA-GENL Practice Question

A representative NVIDIA Generative AI LLMs Associate (NCA-GENL) exam question on Evaluation Metrics & Perplexity. Work through it below, then read why each option is right or wrong.

Short answer

The correct answer is D. Model A assigns higher probability to the test set sequences on average and is a better fit for the domain.

Perplexity measures how well a probability model predicts a sample; lower perplexity means the model assigns higher probability to the observed sequences. Model A's lower perplexity of 12.4 indicates it is a better fit for the domain corpus. Option A incorrectly conflates perplexity with output diversity. Option B is wrong because lower perplexity does not inherently mean overfitting. Option C is a real consideration but the question states both models were fine-tuned on the same corpus, implying comparable evaluation conditions.

The Question

A team is evaluating two autoregressive language models fine-tuned on a domain-specific corpus. Model A achieves a perplexity of 12.4 on the held-out test set, while Model B achieves a perplexity of 18.7. What does this comparison indicate about the models?

AModel B generates more diverse outputs and is therefore preferred for creative tasks
BModel A is overfitting to the training data because lower perplexity always indicates memorization
CPerplexity cannot be compared across two models unless they share the same tokenizer vocabulary
DModel A assigns higher probability to the test set sequences on average and is a better fit for the domainCorrect

Why D is correct

Perplexity measures how well a probability model predicts a sample; lower perplexity means the model assigns higher probability to the observed sequences. Model A's lower perplexity of 12.4 indicates it is a better fit for the domain corpus. Option A incorrectly conflates perplexity with output diversity. Option B is wrong because lower perplexity does not inherently mean overfitting. Option C is a real consideration but the question states both models were fine-tuned on the same corpus, implying comparable evaluation conditions.

Why the other options are wrong

Option A: Model B generates more diverse outputs and is therefore preferred for creative tasks

Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Option B: Model A is overfitting to the training data because lower perplexity always indicates memorization

Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Option C: Perplexity cannot be compared across two models unless they share the same tokenizer vocabulary

Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.

Key idea: Evaluation Metrics & Perplexity

Perplexity measures how well a probability model predicts a sample; lower perplexity means the model assigns higher probability to the observed sequences. Model A's lower perplexity of 12.4 indicates it is a better fit for the domain corpus. Option A incorrectly conflates perplexity with output diversity. Option B is wrong because lower perplexity does not inherently mean overfitting. Option C is a real consideration but the question states both models were fine-tuned on the same corpus, implying comparable evaluation conditions. On the NCA-GENL exam, questions in the "Experimentation and Evaluation" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.

Ready to see how you'd score?

Take the free practice quiz and find out which NVIDIA Generative AI LLMs Associate domains you need to focus on. No signup required.

Take the free quizSee all 500+ practice questions

Practice 5 similar questions

Same cert, same or adjacent domain. Use these after reviewing the explanation.

1Machine Learning & Supervised Learning — NCA-GENL Practice QuestionCore ML and AI Knowledge2Langchain & Python Libraries — NCA-GENL Practice QuestionSoftware Development for LLMs3Data Preprocessing & CuDF — NCA-GENL Practice QuestionData Analysis and Visualization4Bias Mitigation & Fairness — NCA-GENL Practice QuestionTrustworthy AI5Transformer & Attention Mechanism — NCA-GENL Practice QuestionCore ML and AI Knowledge

Related NVIDIA Generative AI LLMs Associate Practice Questions

Machine Learning & Supervised Learning — NCA-GENL Practice Question→Langchain & Python Libraries — NCA-GENL Practice Question→Data Preprocessing & CuDF — NCA-GENL Practice Question→