Glossary · Models & inference
Attention
A mechanism that forms contextual representations by comparing query vectors with key vectors, normalizing the resulting scores, and using them to combine value vectors. Masks, position rules, or sparse patterns can restrict which positions participate.
Why it matters
Attention lets a model route information between sequence positions, but it does not by itself explain or prove what the model understood.
Common confusion
Attention weights are computation coefficients, not a faithful explanation of model reasoning.
Related terms
Sources
Browse the learning paths to see this term in context — every lesson is free to read.