Compute Efficiency

TL;DR for operators The usual AI strategy story is simple: whoever spends the most on compute owns the future. The paper behind this article makes a more awkward claim: under current language-model scaling assumptions, massive compute advantage may be a temporary lead, not a permanent moat.1 The mechanism is not magic. It is diminishing returns. Chinchilla-like scaling laws imply that each additional unit of training compute buys a smaller reduction in loss. Meanwhile, hardware improvement and algorithmic progress are shared forces. They do not only help the largest labs. They also make yesterday’s “small” budget more capable. The result is a curve where frontier models pull ahead, peak in relative advantage, and then become less distinguishable from cheaper models. ...

TL;DR for operators Search-contempt is not “AI plays worse so it learns more”. That would be the lazy interpretation, and business strategy already has enough lazy interpretations wearing expensive shoes. The paper introduces a hybrid MCTS method for AlphaZero-like self-play systems. It behaves like standard PUCT search for the player to move, but at opponent nodes it eventually freezes the opponent’s visit distribution after a threshold, $N_{scl}$, and samples from that frozen distribution rather than constantly updating it toward stronger play.1 The effect is subtle but important: the system stops assuming the opponent will always improve its response with more search. ...