Collective communication

The all-reduce / all-gather / broadcast primitives that synchronize tensors across many accelerators, forming the coordination substrate of every distributed inference or training run.

grounded in: Trend theme 'Distributed / decentralized AI compute' (Mesh LLM meshing inference across peers) plus the existing tensor-parallelism/pipeline-parallelism concepts, all of which rest on collective ops.