Models
Attention
Attention is the mechanism that lets a model weigh how much each token in the input matters when producing the next one.
Also written: self-attention, attention mechanism
Every token can attend to every other token, and the model learns which relationships matter. This is what allows a pronoun to resolve to a noun forty words earlier.
The cost is quadratic in sequence length: doubling the context roughly quadruples the attention computation. Long-context models use various optimisations to soften that, but the underlying pressure is why long prompts are slow as well as expensive.
In practice
Attention concentrates at the start and the end of a context. That is not a quirk to work around, it is a layout instruction: framing and rules go at the top, the decisive material and the question go at the bottom, and anything buried in the middle of a long document is the part most likely to be missed.
Common questions
Does the model read the whole prompt equally?
No. Attention concentrates at the beginning and the end of a context. That makes prompt layout a real decision: framing and rules at the top, the decisive material and the question at the bottom.
Why does moving text change the answer?
Because position affects how strongly the model attends to it. The same fact in the middle of a long document and immediately before the question are not equally likely to be used.