raghu@dark-factory :~/kb/expert-parallelism $ cat

Expert parallelism

A parallelism strategy that shards a Mixture-of-Experts model's experts across devices so each accelerator hosts a subset, routing tokens to the device holding their chosen expert.

grounded in: Established MoE-serving technique (the fourth axis of 4D parallelism alongside data/tensor/pipeline), extends the system's vLLM distributed-inference stack referenced in doctrine and the `moe` concept

Connected concepts

Mixture-of-Experts, Distributed inference, Tensor Parallelism, Pipeline parallelism, Collective communication

Explore it live in the knowledge graph →