Reinforcement-Learning

TL;DR for operators A code agent can produce two patches that pass the same tests while differing sharply in precision, scope, side effects, and consistency with the repository. Gagar1 keeps executable tests as the success gate, but adds an agentic grader to rank the quality of successful implementations. The less obvious contribution is what happens after grading. Simply reducing the reinforcement assigned to weaker successful trajectories also reduces the group’s total positive credit. In the paper’s ablation, that design was accompanied by much faster growth in policy entropy and rollout length. Gagar instead redistributes positive advantage so better successful trajectories receive more relative credit while the original positive-credit total is preserved in the exact formulation. ...

TL;DR for operators Wang et al. study a deployment problem that is easy to underestimate: an RL-trained agent can work well before compression or modification and then fail badly after its policy is perturbed.1 Their proposed method, Stable Perturbation-Robust Policy Optimization (SPrPO), trains the agent under perturbations while allocating less noise to locally sensitive FFN channels and adjusting total perturbation strength using a gradient-based improvement proxy. ...

TL;DR for operators RL post-training can make a model better and longer at the same time. The usual response is to add a length penalty or bonus to the reward, but that changes the signal from which the optimizer decides which sampled responses to reinforce and which to suppress. Weng and colleagues’ QGLAS method1 separates those roles. Quality determines the directional learning signal first. Length can then increase the reinforcement magnitude only for shorter responses that quality already favors. ...

TL;DR for operators Training one model across mathematics, code, instruction following, and chat creates a sequencing problem as well as an allocation problem. An update can improve the capability currently receiving optimization pressure while partially reversing output changes produced by the preceding update. A training run can therefore look relatively benign when gradients are inspected at one checkpoint yet still accumulate damage through the order in which updates are realized. ...

TL;DR for operators Giving a decision-support model access to more knowledge is not the same as making it use knowledge well. In the controlled comparisons reported by RareDx, static retrieval can perform worse than direct inference, phenotype-aware structured retrieval can materially improve diagnostic coverage, and a validation-selected router can outperform using one inference strategy everywhere. ...

TL;DR for operators An optimization assistant can generate executable solver programs for many business problems, but expert-verified formulations are expensive and a trajectory-level success signal does not reveal which modeling section was wrong. That makes solver execution useful not just as a final check, but as a potential source of training supervision. ...

TL;DR for operators A tool agent may need to read records, retrieve identifiers, inspect intermediate state, and only then issue the write that completes a task. If the verifier scores only that final write, a seemingly precise per-turn reward can assign learning credit to one visible action while leaving the prerequisite chain unsupervised. ...

TL;DR for operators After a GUI agent sends a message, edits a document, moves a file, changes a setting, or searches for information, someone—or something—has to decide whether the instruction was actually completed correctly. It is tempting to treat that decision as mainly a model-capability problem: use a stronger vision-language model, show it more screenshots, or improve the prompt. The evidence in Task-Adaptive Rubrics for GUI Reward Modeling suggests another failure point comes earlier. The verifier first needs an adequately specified definition of success.1 ...

TL;DR for operators A synthetic-data budget creates a design choice: generate examples that resemble deployment tasks, or construct simpler exercises that isolate the operations a model will repeatedly need. SpatialBlock1 provides evidence for the second strategy. In a matched Qwen2.5-VL-3B experiment, synthetic training on conventional relative-direction and relative-distance questions improves those same in-domain tasks. But the SpatialBlock curriculum performs better on several external evaluations: 49.1 versus 32.8 on MindCube, 28.1 versus 25.5 on MMSI-Bench, and 46.4 versus 43.6 on MMMU. ...

TL;DR for operators RLVR teams face a familiar trade-off: stronger constraints can stabilize training, but constraining the response distribution too tightly can also suppress useful exploration. The paper identifies a second stability problem that response-side KL alone can miss. The same model being optimized also assigns probabilities to the training queries themselves, and those probabilities can shift substantially even when the dataset stays fixed. ...