Prefill/Decode Disaggregation

An inference architecture that runs the compute-bound prefill phase and the memory-bandwidth-bound decode phase on separate hardware pools, streaming the KV cache between them to optimize each independently.

grounded in: Established technique in modern distributed serving stacks (vLLM/production inference); grounded in the 'Distributed / decentralized LLM compute' HN trend theme (2026-07-12) of splitting inference acr

Connected concepts

KV Cache, Continuous batching, Distributed inference, Memory-bandwidth-bound decode, Inference economics