Opening — Why this matters now For years, scaling AI meant one thing: train bigger models, then freeze them. At inference time, we search harder, sample wider, and hope brute force compensates for epistemic limits. This paper challenges that orthodoxy. It argues—quietly but decisively—that search alone is no longer enough. If discovery problems are truly out-of-distribution, then the model must be allowed to learn at test time. ...
Opening — Why this matters now LLMs have learned to speak fluently. They can reason passably. Some can even plan. Yet most of them remain trapped in an oddly artificial condition: they think, but they cannot act. The latest wave of agent frameworks tries to fix this with tools, APIs, and carefully curated workflows. But a quieter idea is emerging underneath the hype—one that looks less like prompt engineering and more like infrastructure. ...
Opening — Why this matters now Neural theorem proving has entered its industrial phase. With reinforcement learning pipelines, synthetic data factories, and search budgets that would make a chess engine blush, models like DeepSeek‑Prover‑V1.5 are widely assumed to have internalized everything there is to know about formal proof structure. This paper politely disagrees. Under tight inference budgets—no massive tree search, no thousand-sample hail‑Mary—the author shows that simple, almost embarrassingly old‑fashioned structural hints still deliver large gains. Not new models. Not more data. Just better scaffolding. ...
Opening — Why this matters now As reinforcement learning (RL) systems inch closer to real-world deployment—robotics, autonomous navigation, decision automation—a quiet assumption keeps slipping through the cracks: that remembering is enough. Store the past, replay it when needed, act accordingly. Clean. Efficient. Wrong. The paper Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learning dismantles this assumption with surgical precision. Its core claim is blunt: agents that merely retain information fail catastrophically once the world changes. Intelligence, it turns out, depends less on what you remember than on what you are able to forget. ...
Opening — Why this matters now GUI agents are getting smarter in all the wrong ways. Model sizes grow. Benchmarks inch upward. Training datasets balloon into the tens of millions of annotated clicks. Yet in real interfaces—dense IDEs, CAD tools, enterprise dashboards—agents still miss the obvious. Not because they cannot reason, but because they don’t know where to look. ...
Opening — Why this matters now Tool-using agents are no longer a novelty. They are quietly becoming the default interface between LLMs and the real world: APIs, databases, search engines, execution environments. Yet most reinforcement learning pipelines still behave as if every step in a trajectory deserves the same bonus. That assumption was tolerable when tasks were short. It collapses when agents think, call tools, fail, retry, and recover over ten or more turns. ...
Opening — Why this matters now Multimodal AI has spent the last two years narrating its thoughts like a philosophy student with a whiteboard it refuses to use. Images go in, text comes out, and the actual visual reasoning—zooming, marking, tracing, predicting—happens offstage, if at all. Omni-R1 arrives with a blunt correction: reasoning that depends on vision should generate vision. ...
Opening — Why this matters now Multi-agent LLM systems are having a moment. From collaborative coding bots to diagnostic committees and AI tutors, orchestration is increasingly the default answer to hard reasoning problems. But there’s an inconvenient truth hiding behind the demos: training multi-agent systems with reinforcement learning is expensive, unstable, and often counterproductive. ...
Opening — Why this matters now LLM agents are no longer failing because they cannot reason. They fail because they are trained in worlds that are too small, too brittle, or too artificial to matter. As agents are pushed toward real-world tool use—databases, APIs, enterprise workflows—the limiting factor is no longer model size, but environment quality. This paper introduces EnvScaler, a framework arguing that if you want general agentic intelligence, you must first scale the worlds agents inhabit. ...
Opening — Why this matters now Autonomous agents are very good at talking about tasks. They are far less competent at actually doing them—especially when “doing” involves clicking the right icon, interpreting a cluttered interface, or recovering gracefully from failure. GUI agents, in particular, suffer from a chronic problem: once they fail, they either repeat the same mistake or forget everything they once did right. ...