LLM Agents

TL;DR Shachi is a modular methodology for building LLM-driven agent-based models (ABMs) that replaces ad‑hoc prompt spaghetti with four standardized cognitive components—Configs, Memory, Tools, and an LLM reasoning core. The result: agents you can port across environments, benchmark rigorously, and use to study nontrivial dynamics like tariff shocks with externally valid outcomes. For enterprises, Shachi is the missing method for turning agent demos into decision simulators. Why this paper matters to operators (not just researchers) Most enterprise “agent” pilots die in the gap between a clever demo and a reliable simulator that leaders can trust for planning. Shachi closes that gap by: ...

When we measure a marathon by who crosses the line, we ignore how they ran it. For LLM agents that operate through tool calls—editing a CRM, moving a robot arm, or filing a compliance report—the “how” is the difference between deployable and dangerous. Today’s paper introduces CORE: Full‑Path Evaluation of LLM Agents Beyond Final State, a framework that scores agents on the entire execution path rather than only the end state. Here’s why this matters for your roadmap. ...

Thesis: Give an LLM agent freedom and a memory, and it won’t idle. It will reliably drift into one of three meta-cognitive modes. If you operate autonomous workflows, these modes are your real defaults during downtime, ambiguity, and recovery. Why this matters (for product owners and ops) Most agent deployments assume a “do nothing” baseline between tasks. New evidence says otherwise: with a continuous ReAct loop, persistent memory, and self-feedback, agents self-organize—not randomly, but along three stable patterns. Understanding them improves incident response, UX, and governance, especially when guardrails, tools, or upstream signals hiccup. ...

TL;DR Why this paper matters: It shows that how you search matters more than how big your model is for multi‑step, tool‑using analytics. With a notebook‑grounded dataset (NbQA) and value‑guided search, mid‑size open models rival GPT‑4o–based agents on a leading data‑analysis benchmark. What’s new: (1) NbQA, a large corpus of real Jupyter tasks with executable multi‑step solutions; (2) JUPITER, a planner that treats analysis as a tree search over “thought → code → output” steps, guided by a learned value model. Why you should care (operator’s view): This blueprint turns flaky “Code Interpreter”-style sessions into repeatable playbooks—fewer dead ends, more auditable steps, and better generalization without paying for the biggest model. The core idea: analytics as a search tree Most LLM data‑analysis failures come from branching mistakes: choosing the wrong intermediate step, compounding errors, and wasting tool calls. JUPITER reframes the whole exercise as search over notebook states. Each node is a concrete state—accumulated thoughts, code, and execution outputs. The system expands only a few promising branches and prunes the rest using a value model trained on successful and failed trajectories. ...

The quick take Most debates about “diminishing returns” fixate on single‑step metrics. This paper flips the lens: if your product’s value depends on how long a model can execute without slipping, then even small per‑step gains can produce super‑linear increases in the task length a model can finish. The authors isolate execution (not planning, not knowledge) and uncover a failure mode—self‑conditioning—where models become more likely to err after seeing their own past errors. Reinforcement‑learned “thinking” models largely bypass this and stretch single‑turn execution dramatically. ...

TL;DR Plan‑then‑Execute (P‑t‑E) agents separate strategy from action: a Planner writes a machine‑readable plan; an Executor carries it out. This simple split dramatically improves predictability, cost control, and—crucially—security. Hardened correctly (least‑privilege tools, sandboxed code, human sign‑offs), P‑t‑E becomes an enterprise‑grade pattern rather than a lab demo. Why today’s agents need a spine, not vibes Reactive patterns like ReAct feel nimble because they “think, act, observe, repeat.” But that short‑horizon loop is exactly what makes them fragile in production: they meander, retry the same failing step, and are easy to hijack by indirect prompt injection embedded in web pages or PDFs. P‑t‑E locks the control‑flow before the agent ingests untrusted data. The plan becomes an auditable artifact and the execution stage can be cheap, parallel, and tightly permissioned. ...

If you’ve been tempted to A/B‑test a marketing idea on thousands of synthetic “customers,” read this first. A new study introduces a dead‑simple but devastating test for LLM‑based agents: ask them to first state their internal stance (preference) and their openness to persuasion, then drop them into a short dialogue and check whether their behavior matches what they just claimed. That’s it. If agents are believable stand‑ins for people, the conversation outcome should line up with those latent states. ...

Modern retail planning isn’t a spreadsheet; it’s a loop. A new supply‑chain agent framework—deployed at JD.com’s scale—treats planning as a closed‑loop system: gather data → generate plans → execute → diagnose → correct → repeat. That shift from “one‑and‑done forecasting” to continuous replanning is the core idea worth copying. What’s actually new here Agentic decomposition around business intents. Instead of dumping a vague prompt into a model, the system classifies the operator’s request into three intent families: (1) inventory turnover & diagnostics, (2) in‑stock monitoring, (3) sales/inventory/procurement recommendations. Each intent triggers a structured task list rather than ad‑hoc code. Atomic analytics, not monoliths. The execution agent generates workflows as chains of four primitives—Filter → Transform → Groupby → Sort—and stitches them with function calls to vetted business logic. This keeps code inspectable, traceable, and reusable. Dynamic reconfiguration. After every sub‑task, observations feed back into the planner, which prunes, reorders, or adds steps. The output isn’t a static report; it’s a plan that learns while it runs. Why it matters for operators (not just researchers) Traditional MIP‑heavy or rule‑based planning works well when the world is stationary and well‑specified. Retail isn’t. Promotions, seasonality, logistics bottlenecks, supplier constraints—these create moving objective functions. The agentic design here bakes in: ...

Enterprise buyers love what agents can do—and fear what they might do. Meta‑Policy Reflexion (MPR) proposes a middle path: keep your base model frozen, but bolt on a reusable, structured memory of “what we learned last time” and a hard admissibility check that blocks invalid actions at the last mile. In plain English: teach the agent house rules once, then make sure it obeys them, everywhere, without re‑training. The big idea in one slide (text version) What it adds: a compact, predicate‑like Meta‑Policy Memory (MPM) distilled from past reflections (e.g., “Never pour liquid on a powered device; unplug first.”) ...

The Big Idea Operating systems have always struggled with a silent mismatch: the kernel’s scheduler doesn’t know what your application actually wants. SchedCP proposes a clean solution—turn scheduling into a semantic control plane. AI agents reason about what a workload needs; the system safely handles how to observe and act via eBPF-based schedulers. This division keeps LLMs out of the hot path while letting them generate and refine policies that actually fit the job. ...