raghu@dark-factory :~/kb/expert-offloading $ cat

Expert offloading

Serving a Mixture-of-Experts model on limited memory by keeping only the currently-active experts resident and streaming the inactive expert weights from CPU RAM or disk on demand.

grounded in: HN front-page theme (2026-07-16): fixation on running large open models cheaply without GPUs and storing weights efficiently; extends the existing moe + model-offloading concepts.

Connected concepts

Mixture-of-Experts, Model offloading, Expert parallelism, CPU inference serving, Memory-mapped weights

Explore it live in the knowledge graph →