Cover image

Not Every Passing Patch Deserves the Same Credit

TL;DR for operators A code agent can produce two patches that pass the same tests while differing sharply in precision, scope, side effects, and consistency with the repository. Gagar1 keeps executable tests as the success gate, but adds an agentic grader to rank the quality of successful implementations. The less obvious contribution is what happens after grading. Simply reducing the reinforcement assigned to weaker successful trajectories also reduces the group’s total positive credit. In the paper’s ablation, that design was accompanied by much faster growth in policy entropy and rollout length. Gagar instead redistributes positive advantage so better successful trajectories receive more relative credit while the original positive-credit total is preserved in the exact formulation. ...

October 12, 2026 · 8 min · Zelina
Cover image

Shorter Without Flipping the Signal: QGLAS Reworks Length Control in Open-Ended RL

TL;DR for operators RL post-training can make a model better and longer at the same time. The usual response is to add a length penalty or bonus to the reward, but that changes the signal from which the optimizer decides which sampled responses to reinforce and which to suppress. Weng and colleagues’ QGLAS method1 separates those roles. Quality determines the directional learning signal first. Length can then increase the reinforcement magnitude only for shorter responses that quality already favors. ...

October 7, 2026 · 7 min · Zelina
Cover image

When the Reward Is Right but the Incentive Is Wrong

TL;DR for operators A preference score can rank responses correctly and still be the wrong signal to feed directly into an alignment system. Wang et al. show why: when deployment deliberately keeps the aligned model close to its base behavior, the base model’s own response probabilities continue to influence what gets generated. Their Stackelberg Reward Shaping framework therefore changes the reward landscape rather than simply increasing reward strength.1 ...

September 14, 2026 · 8 min · Zelina
Cover image

Shape the Signal, Keep the Objective: MeRLa’s Bet on Reusable RLHF Rewards

TL;DR for operators RLHF teams usually have two obvious levers: improve the reward model or improve the policy optimizer. Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback introduces a third.1 MeRLa learns an additional task-aware reward signal across auxiliary tasks, freezes it, and adds it to the existing reward during subsequent policy optimization. ...

August 23, 2026 · 7 min · Zelina