When Another Training Round Stops Buying Learning

TL;DR for operators

Closed-loop self-training should not be treated as productive merely because the system is still generating examples and updating weights. Wang et al. propose measuring whether each new round contains patterns that were not already predictable from the previous round, then using that signal to decide both which samples deserve gradient weight and whether another round is worth running at all.¹

In their reported experiments, the signal approaches zero before held-out accuracy peaks. Training well beyond that point is associated with falling accuracy and contraction in solution diversity. Their ATRI method combines sample-level information-gain weighting with a round-level stopping rule and outperforms the strongest aggregate baseline in the main Qwen3-4B and Qwen3-8B comparisons.

For teams running autonomous synthetic-data loops, the practical implication is a control-policy change: continuation should depend on whether the loop is still supplying learnable information. Once that signal is exhausted, the next dollar of compute may be better spent on targeted external data than on another internally generated round.

Training can continue after learning has stopped

A self-improvement pipeline can look healthy for surprisingly mundane reasons. New questions are being generated. New answers are being produced. Optimization steps complete. The model changes.

None of those facts establishes that the next round contains information the system has not already absorbed.

That distinction matters because another round has three costs: generation, optimization, and delayed checkpoint selection. If useful learning has saturated before validation accuracy visibly deteriorates, an organization can spend several expensive rounds discovering after the fact that its best model existed earlier.

The paper’s preliminary Qwen3-4B-Base experiment on MATH illustrates the problem. Its proposed round-level signal falls to approximately zero at round 3. Accuracy peaks at round 4 and then declines. Reward variance, policy entropy, distinct-n-gram ratio, and 1-self-BLEU do not supply a comparably consistent warning.

The operational question therefore shifts from “Is the loop still training?” to “Is the loop still receiving information it can use?”

A small proxy measures what changed between rounds

The authors call their diagnostic learnable information gain.

At each round, they fine-tune a small proxy language model for one epoch on the previous round’s question-answer data. That proxy is not trying to solve the benchmark. It is a measuring instrument: a compact approximation of what the previous round’s data distribution looked like.

For a sequence $x$, the proxy assigns a token-normalized encoding cost:

The round-level score then compares the proxy’s average cost on the current and previous datasets:

If the new round is harder for a proxy fitted to the previous round to predict, $C_t$ is positive. As successive rounds become increasingly similar to what the loop has already produced, that difference contracts toward zero.

The theoretical interpretation is stronger under an important condition. If the proxy exactly matches the previous-round distribution, the authors derive:

So the score reflects both distributional change and the change in entropy between rounds. The exact identity depends on an exact-fit assumption; the practical system uses a finite-capacity proxy and therefore only approximates that reference.

This matters because the metric is not simply perplexity, entropy, or generic diversity. It is explicitly relative to what the previous round already represented.

ATRI turns the diagnostic into a training control

The paper’s method, ATRI, intervenes at two levels.

Within a round, samples receive weight only for positive estimated information gain. Questions or answers that look no more informative than the previous-round baseline contribute no information-gain weight. Across rounds, the system stops when $C_t$ remains below

for two consecutive rounds.

The distinction between these mechanisms is useful. Sample weighting answers where the next gradient should go. Early stopping answers whether another closed-loop round should exist at all.

The main benchmark results suggest both contribute, but stopping carries more of the measured effect. On Qwen3-4B-Base, full ATRI achieves a Math AVG of 53.55 and Overall AVG of 47.39. Removing weighting while retaining stopping reduces those to 52.49 and 46.31. Retaining weighting without stopping reduces them further to 51.27 and 45.16. Removing both produces 50.18 and 44.05.

Against the strongest aggregate baseline, R-Diverse, ATRI improves Math AVG from 52.56 to 53.55 and Overall AVG from 46.20 to 47.39 on Qwen3-4B-Base. On Qwen3-8B-Base, the corresponding comparison is 56.49 to 58.94 for Math AVG and 50.19 to 52.75 overall.

Those are benchmark gains, not proof that the same stopping rule transfers unchanged to every self-training system. But the ablation gives the operational claim a clearer shape: avoiding low-value rounds appears to matter at least as much as reallocating weight inside a round.

Low information gain precedes contraction, not instant collapse

A near-zero $C_t$ should not be read as a theorem that performance collapses immediately.

The reported pattern is sequential. Information gain falls first. Accuracy then plateaus or peaks. Continued training in the low-gain regime is later associated with performance loss and narrower solution behavior.

In the Vanilla Qwen3-4B-Base/MATH trajectory, pass@8 peaks at 75.4 at round 4 and falls to 55.6 by round 15. Over the same comparison, solution-path entropy falls from 1.846 to 1.173, while the number of distinct correct paths falls from 2.643 to 1.412.

That is more consequential than simple stagnation. The later checkpoints are not merely failing to improve; they are producing a narrower set of successful solution paths while accuracy deteriorates.

The paper describes this as a three-phase empirical lifecycle: improvement while information gain is positive, saturation as it approaches zero, and degeneration under prolonged low-gain training. The lifecycle is an observed regularity in the reported settings, not a universal theorem.

The monitor is cheap enough to change the economics

A diagnostic has limited operational value if measuring the system costs nearly as much as running it.

Here the proxy layer is comparatively small. On Qwen3-8B-Base, the paper reports 87.8 PFLOPs per Vanilla round and 89.7 PFLOPs with ATRI, a 2.2% increase. Measured GPU-hour overhead falls from 4.1% at the 4B scale to 1.3% at 14B because the fixed-size proxy remains approximately constant while the main loop becomes more expensive.

The paper also applies $C_t$ passively to six 15-round baseline trajectories. Average MATH accuracy at the final round is 39.01. Oracle checkpoint selection reaches 47.15. Passive information-gain stopping reaches 46.32 without using held-out validation accuracy.

For an operator, that comparison is more interesting than the metric itself. A low-cost control layer recovered checkpoints close to those selected with hindsight while avoiding the assumption that validation accuracy will always be available, reliable, or cheap to query after every round.

When the closed loop is exhausted, new information has to come from somewhere

Stopping solves waste. It does not create another source of learning signal.

The paper therefore tests an optional external-data phase. Its directional extension asks two separate questions about a candidate sample: is it novel relative to what the model has already absorbed, and does it resemble cases the current model gets wrong?

That distinction prevents high surprise from being mistaken for usefulness. In the Qwen3-4B ablation setting, adding the external phase increases Math AVG from 53.55 to 55.67 and Overall AVG from 47.39 to 49.60.

Cognaptus interprets this as a practical handoff rule for autonomous training systems. Once internally generated information gain has saturated, the next optimization problem changes from generating more of the same loop to sourcing data that is both new and connected to observed failure modes.

That inference assumes a useful external candidate pool exists. ATRI cannot manufacture genuinely new information when neither the model’s own loop nor external sources contain it.

Where operators should keep the boundary

The strongest use case supported by the paper is closed-loop iterative self-training with meaningful round boundaries or equivalent data windows.

Several boundaries remain material. The experiments use majority-vote or ground-truth verification rather than learned reward models, so behavior under reward hacking or learned-verifier drift is not established. The default threshold $\tau=0.1C_1$ is empirically selected rather than theoretically optimal. Task shifts require refitting the proxy: in the reported MATH-to-MBPP test, carrying the old proxy forward causes the alarm to arrive after the new task’s performance peak, while refitting restores the one-round-early warning pattern.

Generality is also finite. The lifecycle and alarm behavior were tested across reported Qwen and Llama models, MATH and MBPP tasks, Pythia and OPT proxies, and six training algorithms. That is meaningful robustness evidence, but not a universal law of self-training.

The operational principle is narrower and more defensible: do not equate another update with another unit of learning.

For teams building synthetic-data or self-improvement pipelines, each round should therefore face a continuation test. If the new data remain informationally distinct, keep training. If the loop has become predictable to itself, stop spending compute on repetition, preserve the stronger checkpoint, and look for information the system does not already generate.

Cognaptus: Automate the Present, Incubate the Future.