Continuous batching

An LLM-serving scheduler that adds and evicts sequences from a running GPU batch at the token level, keeping utilization high under mixed-length request streams.

grounded in: vLLM is the system's local inference server (`vllm` in the ontology + `spark`/`spark_stats.py` GB10 2-node cluster telemetry); continuous batching is vLLM's foundational serving technique.

Connected concepts

vLLM / PagedAttention, Distributed inference, Prefix/KV-cache reuse