KV-Cache Quantization

Compressing or quantizing the transformer KV cache (per-token, per-channel, or via eviction) to cut the memory-bandwidth and capacity cost of long-context inference.

grounded in: Trend theme 'LLM interpretability and efficient training... research interest shifting toward cheaper long-context kernels', plus this system's local vLLM inference stack (vllm) where KV cache dominat