Scaling laws

Empirical power-law relationships predicting how model loss falls with training compute, dataset size, and parameter count, used to allocate compute optimally.

grounded in: Trend theme 'Recursive self-improvement & AI economics' (the economics of scaling AI systems) grounded on the established foundational ML result behind compute budgeting.

Connected concepts

Scaling diminishing returns, Inference economics, Capability–cost frontier, Mixture-of-Experts, Knowledge distillation