Transformer & Attention Mechanism — NCA-GENL Practice Question
A representative NVIDIA Generative AI LLMs Associate (NCA-GENL) exam question on Transformer & Attention Mechanism. Work through it below, then read why each option is right or wrong.
Short answer
The correct answer is B. The self-attention mechanism that computes relevance scores between all pairs of tokens in the sequence.
The self-attention mechanism computes attention scores between every pair of tokens, allowing the model to directly relate "she" to "Dr. Martinez" regardless of their distance in the sequence. This is the transformer's primary mechanism for capturing long-range dependencies. Feed-forward layers (A) process each position independently and do not model inter-token relationships. Positional encoding (C) provides order information but does not itself resolve dependencies. Layer normalization (D) stabilizes training but does not perform relational reasoning.
The Question
An engineer is analyzing why a transformer-based language model attends to a pronoun's antecedent in a long passage. During inference, the model correctly resolves "she" to "Dr. Martinez" mentioned 200 tokens earlier. Which component of the transformer architecture is most directly responsible for this long-range dependency resolution?
Why B is correct
The self-attention mechanism computes attention scores between every pair of tokens, allowing the model to directly relate "she" to "Dr. Martinez" regardless of their distance in the sequence. This is the transformer's primary mechanism for capturing long-range dependencies. Feed-forward layers (A) process each position independently and do not model inter-token relationships. Positional encoding (C) provides order information but does not itself resolve dependencies. Layer normalization (D) stabilizes training but does not perform relational reasoning.
Why the other options are wrong
Option A does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Option C does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Option D does not satisfy the requirement in the scenario. Review the explanation above: the correct choice (B) is the only one that fully meets every constraint stated in the question.
Key idea: Transformer & Attention Mechanism
The self-attention mechanism computes attention scores between every pair of tokens, allowing the model to directly relate "she" to "Dr. Martinez" regardless of their distance in the sequence. This is the transformer's primary mechanism for capturing long-range dependencies. Feed-forward layers (A) process each position independently and do not model inter-token relationships. Positional encoding (C) provides order information but does not itself resolve dependencies. Layer normalization (D) stabilizes training but does not perform relational reasoning. On the NCA-GENL exam, questions in the "Core ML and AI Knowledge" domain test whether you can map a scenario's constraints to the right choice. Read the requirement carefully, eliminate options that violate any single constraint, and pick the one that satisfies all of them with the least operational overhead.
Ready to see how you'd score?
Take the free practice quiz and find out which NVIDIA Generative AI LLMs Associate domains you need to focus on. No signup required.
Practice 5 similar questions
Same cert, same or adjacent domain. Use these after reviewing the explanation.