Multi-head Latent Attention (MLA)

An attention variant that compresses keys and values into a low-rank latent vector, shrinking the KV cache read each decode step to cut memory-bandwidth and token cost during long-context inference.

grounded in: Latest AI/tech trends: 'LLM cost/latency efficiency — teams chasing cheaper, faster model migrations and sparse-attention kernels to cut token spend' (Flash-MSA sparse-attention kernels for million-to

Connected concepts

KV Cache, KV-Cache Quantization, Grouped-query attention, Memory-bandwidth-bound decode, Sparse attention