Encoders and Decoders: Where Q, K, and V Come From
Same attention math, two job titles: the encoder and the decoder.
Image backend outage note: the image generator (qwen/qwen-image-3 via OpenRouter) was funded but rendered a degraded plate on every roll this run (title-word and bullet garble, prompt-scaffolding text on the sticky note — the same funded-primary degradation signature recorded on Oct 10–12). Per the outage contract this is a text-only card; the full teaching content is below, unchanged.
1Self-attention: all vectors from one sequence
The encoder's job — context built for understanding
Queries, Keys and Values all come from the same sequence.
Every token attends to every other token in one block.
BERT reads both directions; GPT masks the right and reads left-only.
The encoder runs self-attention over the full input — every token gets a context built around it.
Decoder-only GPT (GPT-4 and friends) runs on this alone: no cross-attention anywhere.
2Cross-attention: Q from one, K and V from another
The decoder's second source — context built while generating
The math is identical to self-attention — only the source of the vectors changes.
Queries come from the decoder's own tokens (the target language so far).
Keys and Values come from the encoder's output (the source language).
Translation is the classic shape: while generating the target, the decoder keeps pulling information from the source.
In T5, the decoder's cross-attention is how the source sentence stays reachable mid-generation.
3Where context is actually made
The mechanism a definition slide always drops
The static embedding-table row is just the input — a token's identity, not its meaning here.
Attention is what converts that row into a contextual vector.
new token vector = weighted mix of the
value vectors of everything it attends to
Each token's output is a weighted mix of all the tokens it attends to — context is computed, not looked up.
Encoder: context built for understanding (full input, both directions).
Decoder: context built while generating (causal self-attention over generated tokens + cross-attention to encoder states).
4Two corrections that break the shortcut
The falsification — test any explanation against these
Cross-attention is not the whole decoder. In the original Transformer and in T5, each decoder block has three sub-layers: masked self-attention, then cross-attention, then a feed-forward network.
GPT has no cross-attention at all. Its “context” is simply more tokens in the same sequence — which is exactly why the prompt and the completion can be concatenated into one string and fed through self-attention.
So the decoder-vs-encoder difference is not “attention vs no attention”; it is where the K and V vectors come from.
The line to remember
The table row is input. Attention makes the context.