raghu@dark-factory :~/kb/sliding-window-attention $ cat

Sliding-window attention

A long-context technique that restricts each token's attention to a fixed local window, giving linear cost over sequence length and pairing naturally with retained attention-sink tokens.

grounded in: Trend theme 'cheaper long-context kernels' and the established attention-sink/StreamingLLM lineage already in the graph; a lasting kernel-level efficiency pattern.

Connected concepts

Attention sink, Sparse attention, FlashAttention (IO-aware attention), KV Cache, Ring Attention

Explore it live in the knowledge graph →