Reinforcement Learning

What if the hardest part of finance isn’t prediction, but precision? Behind every real-time portfolio adjustment or split-second options quote lies a giant math problem: solving Ax = b, where A is large, sparse, and often very poorly behaved. In traditional finance pipelines, iterative solvers like GMRES or its flexible cousin FGMRES are tasked with solving these linear systems — be it from a Markowitz portfolio optimization or a discretized Black–Scholes PDE for option pricing. But when the matrix A is ill-conditioned (which it often is), convergence slows to a crawl. Preconditioning helps, but tuning these parameters is more art than science — until now. ...

Handling long documents has always been a source of frustration for large language models (LLMs). From brittle extrapolation hacks to obscure compression tricks, the field has often settled for awkward compromises. But the paper MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent boldly reframes the problem: what if LLMs could read like humans—absorbing information chunk by chunk, jotting down useful notes, and focusing on what really matters? At the heart of MemAgent is a surprisingly elegant idea: treat memory not as an architectural afterthought but as an agent policy to be trained. Instead of trying to scale attention across millions of tokens, MemAgent introduces a reinforcement-learning-shaped overwriteable memory that allows an LLM to iteratively read arbitrarily long documents in segments. It learns—through reward signals—what to keep and what to discard. ...

If the future of reasoning in large language models (LLMs) doesn’t lie in human-tweaked datasets or carefully crafted benchmarks, where might it emerge? According to SPIRAL, a recent framework introduced by Bo Liu et al., the answer is clear: in games. SPIRAL (Self-Play on zero-sum games Incentivizes Reasoning via multi-Agent muLti-turn reinforcement learning) proposes that competitive, turn-based, two-player games can become a reasoning gymnasium for LLMs. It provides an automated and scalable path for cognitive skill acquisition, sidestepping human-curated data and rigid reward functions. ...

Human-AI collaboration is easy to romanticize in theory but hard to operationalize in practice. While reinforcement learning agents have dazzled us in games like Go and StarCraft, they often stumble when asked to cooperate with humans under real-world constraints: imperfect information, ambiguous signals, and no chance to train together beforehand. That’s the realm of ad-hoc teamwork—and the latest paper from Oxford’s FLAIR lab introduces a critical step forward. The Ad-Hoc Human-AI Coordination Challenge (AH2AC2) tackles this problem by leveraging Hanabi, a cooperative card game infamous among AI researchers for its subtle, communication-constrained dynamics. Unlike chess, Hanabi demands theory of mind—inferring what your teammate knows and intends based on sparse, indirect cues. It’s a Turing Test of collaboration. ...

When it comes to language model agents, more minds may not always mean merrier results. Multi-agent reinforcement learning (MARL) promises a flexible path for decomposing and solving complex tasks, but coordinating multiple large language models (LLMs) remains riddled with instability, inefficiency, and memory fragmentation. Enter JoyAgents-R1, a novel framework that proposes an elegant, scalable solution for jointly evolving heterogeneous LLM agents using Group Relative Policy Optimization (GRPO). Developed by researchers at JD.com, JoyAgents-R1 combines memory evolution, policy optimization, and clever sampling strategies to form a resilient multi-agent architecture capable of matching the performance of larger SOTA models with far fewer parameters. ...

1. A Student Who Cracked the Code — But Not the Meaning Imagine a student who aces every test by memorizing the positions of correct answers on multiple-choice sheets. He scores high, earns accolades, and passes every exam — but understands none of the material. His reward system is misaligned: success depends not on learning, but on exploiting test mechanics. Now, replace the student with an AI agent navigating a simulated room guided by language and images. This is the scenario that today’s leading research in Vision-and-Language Reinforcement Learning (RLVR) is grappling with. ...

The GUI Agent Bottleneck: Stuck in Sparse Feedback Training LLM-based GUI agents to complete digital tasks—such as navigating mobile apps or automating workflows—faces a fundamental limitation: reward sparsity. Traditional reward formulations (Outcome Reward Models, or ORMs) provide feedback only at the end of a trajectory. If the task fails, the agent receives zero signal, regardless of how many useful intermediate steps it took. This severely limits credit assignment and slows learning, especially in environments with long action horizons. ...

Business Process Automation (BPA) has long promised leaner operations, improved responsiveness, and higher profitability. But for physical manufacturing, where every parameter shift impacts material use, energy cost, and defect rate, true real-time optimization remains a complex frontier. In a recent paper, researchers presented a compelling DRL-based solution to injection molding optimization that could signal a broader wave of intelligent, profit-driven automation in smart factories. ...

Cool Heads Prevail: Human-in-the-Loop AI for Smarter HVAC Careers Heating, ventilation, and air conditioning (HVAC) systems are often taken for granted—until they fail or run up a massive electricity bill. But in a world facing both climate urgency and rising energy costs, the traditional thermostat just won’t cut it. Enter a novel Human-in-the-Loop (HITL) AI framework that could reshape how HVAC engineers, facility managers, and energy analysts approach their craft. ...

Embodied Intelligence: A Different Kind of Smart Artificial intelligence is no longer confined to static models that churn numbers in isolation. A powerful shift is underway—toward embodied AI, where intelligence is physically situated in the world. Unlike stateless AI models that treat the world as a dataset, embodied AI experiences the environment through sensors and acts through physical or simulated bodies. This concept, championed by early thinkers like Rolf Pfeifer and Fumiya Iida (2004), emphasizes that true intelligence arises from an agent’s interactions with its surroundings—not just abstract reasoning. Later surveys, such as Duan et al. (2022), further detail how modern embodied AI systems blend simulation, perception, action, and learning in environments that change dynamically. ...