Attention
Also: self-attention
A way for a model to weigh links between parts of its input. The weights help it process text. They do not show which claims are true.
The fuller definition
Attention is a mechanism that combines information from input representations using computed weights. In self-attention, positions in a sequence draw information from other positions in that sequence.
- bank
Ambiguous alone
- loan
Useful surrounding context
- Representation
Context-dependent numerical values
A simplified relationship, not a screenshot or a complete design.
A concrete example
In “The bank approved the loan,” surrounding words help represent bank in a financial context.
The distinction that matters
Attention weights are not a dependable explanation of a model's complete reasoning or a human attention span.
Related terms
Transformer
The design most modern language models use. Its trick is letting every word in a passage look at every other word to work out what matters.
FoundationsToken
A piece of text, often part of a word, that the model actually works with. Models read and write tokens, not whole words.
Documents & retrievalEmbedding
A list of numbers used to compare items such as texts. Texts with similar meanings may have similar lists.
Where it is taught
What actually happens when you ask AI a question?Source and further reading
Google · Machine Learning GlossaryThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.