Model Training

Credit Where It's Due: How CAPO Brings Verifiable Precision to LLM Reasoning

TL;DR for operators CAPO is not mainly a paper about “making models reason better” in the usual fog-machine sense. It is about fixing a specific training failure: outcome-only reinforcement learning tells a model whether the final answer was right, but not which part of the reasoning earned or destroyed that outcome. The method uses a stronger off-the-shelf LLM as a generative process reward model, or GenPRM, to inspect a rollout and identify wrong reasoning steps in one pass. Those step-level critiques are then converted into token-level penalties, so the policy update can suppress flawed reasoning segments instead of treating the whole answer as one indivisible blob. The authors test this across Llama-3-1B/3B and Qwen2.5-1.5B/7B backbones, with results showing consistent average gains over SFT, GRPO with rule-based verification, and GRPO with generative outcome reward modelling.1 ...

Backtrack to the Future: How ASTRO Teaches LLMs to Think Like Search Algorithms

TL;DR for operators ASTRO is not another paper saying “make the model think longer” and then acting surprised when token bills become a lifestyle choice. It is more specific: the authors train a non-reasoner Llama model to imitate the procedure of search. The model is taught to explore a wrong path, notice uncertainty, backtrack, and continue from an earlier step — all inside one generated answer. ...

The Reasoning Gymnasium: How Zero-Sum Games Shape Smarter LLMs

TL;DR for operators SPIRAL is not interesting because it teaches language models to play TicTacToe, Kuhn Poker, and negotiation games. That would be charming, but not exactly a boardroom emergency. Its real contribution is showing that adaptive competitive pressure can train reasoning behaviours that transfer beyond the game environment.1 The paper’s central lesson is mechanism-first: self-play creates a moving curriculum. The model does not merely imitate expert trajectories or exploit a fixed opponent. It faces a continuously improving version of itself, so yesterday’s shortcut becomes today’s liability. That pressure appears to produce reusable reasoning patterns: case-by-case analysis, expected value calculation, and pattern recognition. ...