Reinforcement Learning

TL;DR for operators A system can discover agent “skills” that look coherent to humans and still fail to make an agent more capable. That is the useful result of Automating SKILL.md Generation for Computer-Using Agents via Interaction Trajectory Mining.1 The authors build a pipeline that cuts GUI interaction histories into segments, clusters those segments into candidate routines, generates explicit skill specifications, and trains a Qwen3-8B policy with Group Relative Policy Optimization, or GRPO, to compose the resulting skills. ...

TL;DR for operators A fixed rubric is a depreciating training asset. Early in reinforcement learning, it may be too demanding to distinguish one weak answer from another. Later, once the model learns to satisfy it, the same rubric becomes too easy. The score survives; the information content does not. EvoRubrics trains the evaluator alongside the model.1 A Policy LLM produces candidate answers, a Rubric Generator produces candidate evaluation criteria, and an external judge scores every answer against every rubric. The policy is rewarded for satisfying the evolving criteria. The rubric generator is rewarded for producing criteria that separate stronger from weaker answers, cover different dimensions, remain anchored to desired preferences, and help the policy revise its responses. ...

TL;DR for operators MaxProof is not primarily a story about a model suddenly becoming brilliant at mathematics. It is a story about wrapping an imperfect model in a disciplined production process. MiniMax trains M3 to perform three distinct jobs: write proofs, identify concrete errors in proofs, and repair proofs using those critiques. At inference time, MaxProof generates a population of candidate solutions, evaluates them conservatively, preserves competing approaches, applies both targeted patches and broader rewrites, and finally chooses one answer through pairwise comparison.1 ...

TL;DR for operators A new paper, Self-Paced Curriculum Reinforcement Learning for Autonomous Superbike Racing in Simulation, introduces a reinforcement-learning framework for training a superbike agent in VRider SBK, a Unity-based motorcycle racing simulator.1 The useful part is not merely that the model rides faster. The useful part is how the authors turn motorcycle racing into a staged learning problem without hand-writing a long curriculum by committee, which is usually how such things go to die politely. ...

TL;DR for operators A CAD assistant that writes code once and hopes for the best is not an engineering workflow. It is a raffle with syntax highlighting. IterCAD is interesting because it treats CAD generation and editing as an iterative operating loop: read the drawing, generate CadQuery code, execute it in a sandbox, inspect compiler and geometric feedback, revise, and stop only when the model has evidence that the shape is right.1 The paper’s practical contribution is not “AI can design parts now.” That would be the usual confetti cannon, and mercifully not the correct lesson. The better lesson is that useful CAD automation needs closed-loop verification, localized visual grounding, and evaluation metrics that count failures instead of quietly hiding them in the basement. ...

TL;DR for operators AI systems are increasingly being asked to work in messy, high-dimensional environments: long video archives, multilingual evidence, persona-specific retrieval, humanoid motion, physical contact, timing, perception, and real-world deployment. The temptation is familiar: throw a stronger model at the whole thing and hope intelligence leaks out of the parameter count. Charming. Also expensive. ...

TL;DR for operators AI learning is becoming less like “train a bigger model and hope it behaves” and more like operating a controlled capability loop. The first paper in this cluster shows a narrow but important lesson: once a multimodal model has learned useful representations, the final adaptation step should optimize the metric that actually matters, while avoiding damage to the representation underneath.1 The second paper moves the same logic into physical action: an embodied system should connect language-level intention, predicted world change, memory, and executable robot control, not merely map images to motor commands with expensive optimism.2 The third paper zooms out: when agentic AI becomes economically and militarily useful, the real bottleneck includes data centers, accelerators, electricity, water, datasets, and skilled labor.3 ...

TL;DR for operators Memory is not valuable because an agent writes something down. That is called logging. Sometimes it is called “reflection,” if the logging has better branding. The paper Enhancing Software Engineering Through Closed-Loop Memory Optimization introduces MemOp, a framework for software-engineering agents that defines memory utility by downstream impact: a memory is useful only if it improves the agent’s later performance on software tasks.1 The important move is not the existence of Memory.md, nor the idea that past trajectories can be summarized. The important move is the loop: generate memory from an agent trajectory, validate whether that memory improves task performance, reject harmful or redundant memories, and train a memory model using the resulting accepted and rejected examples. ...

TL;DR for operators EV charging optimization has a small, rude problem: the most important variable is often the one the operator does not know. A plugged-in car may leave in twenty minutes or three hours. That difference determines whether the controller can wait for cheap electricity or must charge immediately like an anxious intern with a deadline. ...

TL;DR for operators Most agent failures are blamed on the model because blaming “the model” is emotionally convenient and operationally vague. HarnessX makes a more useful claim: the runtime harness around the model — prompts, tools, memory, control flow, tracing, evaluators, safety checks, and training interfaces — is not scaffolding in the disposable sense. It is part of the system’s intelligence surface.1 ...