PagedAttention

A KV-cache memory-management technique pioneered by vLLM that stores attention keys and values in non-contiguous fixed-size blocks to eliminate fragmentation and raise inference throughput.

grounded in: vLLM is the system's inference stack (existing 'vllm' concept; spark.sh/spark_stats.py 2-node GB10 telemetry); PagedAttention is vLLM's signature KV-cache technique.

Connected concepts

vLLM / PagedAttention, KV Cache, Continuous batching, Memory-bandwidth-bound decode