Glossary · Models & inference
Encoder
A component that transforms input into a representation. A transformer encoder commonly uses non-causal self-attention, subject to any masks, so each position can incorporate context from across the input.
Common confusion
Encoder-only models can produce outputs through task heads even though they are not typically used for autoregressive text generation.
Related terms
Browse the learning paths to see this term in context — every lesson is free to read.