Scaling laws
The empirical finding that a model's loss falls predictably as you add compute, data, and parameters — turning "make it bigger" from a hunch into a forecastable curve.
Scaling laws are the observation that, across orders of magnitude, a language model's error drops smoothly and predictably with more compute, more data, and more parameters. Because the curve is regular, you can forecast how good a bigger model will be before training it — which is why labs pour capital into scale. The open question is where, or whether, the curve finally bends.
Scaling laws reframed AI progress as an engineering forecast rather than a series of lucky architectures. If loss is a predictable function of compute, the rational move is to buy more compute — which is much of what the frontier labs have done.
The debate they fuel: optimists read the unbroken curve as a runway to far more capable systems; sceptics argue the benchmarks that keep improving aren't the ones that matter, and that something other than scale is missing. Both camps are watching the same graph.