Nemo Curator & Semantic Deduplication — NCP-GENL Practice Question
A representative NVIDIA-Certified Professional Generative AI LLMs (NCP-GENL) exam question on Nemo Curator & Semantic Deduplication. Work through it below, then read why each option is right or wrong.
This question is part of the free NVIDIA-Certified Professional Generative AI LLMs practice test.
Short answer
The correct answer is D. Semantic deduplication, which clusters embeddings of each document.
The NeMo Curator documentation says semantic deduplication uses the meaning of the content to find pairs that say the same thing in different words. It generates embeddings, clusters them with K-means, and computes cosine similarity within clusters. Option A matches only character-for-character identical text. Option B finds near-duplicates with small edits, where documents share a lot of overlapping text, which these rewritten items do not. Option C detects repeated paragraphs inside one document, not duplicates across documents.
The Question
A training corpus contains many news items that report the same facts in different wording, rewritten by different outlets. Exact text overlap between these pairs is low. Which NeMo Curator deduplication method targets this kind of duplicate?
Why D is correct
The NeMo Curator documentation says semantic deduplication uses the meaning of the content to find pairs that say the same thing in different words. It generates embeddings, clusters them with K-means, and computes cosine similarity within clusters. Option A matches only character-for-character identical text. Option B finds near-duplicates with small edits, where documents share a lot of overlapping text, which these rewritten items do not. Option C detects repeated paragraphs inside one document, not duplicates across documents.
Why the other options are wrong
Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.
Option B does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.
Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (D) is the only one that fully meets every constraint stated in the question.
Key idea: Nemo Curator & Semantic Deduplication
The NeMo Curator documentation says semantic deduplication uses the meaning of the content to find pairs that say the same thing in different words. It generates embeddings, clusters them with K-means, and computes cosine similarity within clusters. Option A matches only character-for-character identical text. Option B finds near-duplicates with small edits, where documents share a lot of overlapping text, which these rewritten items do not. Option C detects repeated paragraphs inside one document, not duplicates across documents. On the NCP-GENL exam, questions in the "Data Preparation" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which NVIDIA-Certified Professional Generative AI LLMs domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.
Related NVIDIA-Certified Professional Generative AI LLMs Practice Questions
Official sources used
Verify program and exam details at the source before scheduling.