Attention sink

Transformer models dump excess attention onto a few initial tokens, so those tokens' KV-cache entries must be retained to keep long-context generation stable — a mechanistic finding that now guides cache-eviction policy.

grounded in: latest AI/tech trends — 'LLM interpretability: Researchers are moving from black-box use toward causal, mechanistic understanding of model internals'