Adaptive inference serving

Inference servers that profile their live workload and self-tune execution (batching, caching, kernel and route selection) at runtime, getting faster the longer they run.

grounded in: latest AI/tech trends theme 'Inference that self-optimizes at runtime' plus the notable item Reame, a CPU inference server that gets faster the longer it runs