The agent finished the task. That is not the same as doing the task. Inbox sorted. Calendar updated. Report generated. Customer record changed. Dashboard refreshed. For a demo, that is usually enough. The screen shows a plausible answer, the final artifact looks tidy, and everyone politely pretends the agent must have followed the correct path because the output did not immediately burst into flames. ...
The checklist problem that exposes the model Checklist tasks look boring. That is exactly why they are useful. Ask an LLM to write a formal email under 50 words, include one required term, avoid another term, and return the result as JSON. None of this sounds intellectually difficult. No theorem proving. No multimodal reasoning. No dramatic benchmark leaderboard screenshot. Just instructions. ...
Memory sounds simple until a business actually needs it. A sales agent should remember what the client objected to last month. A customer-support agent should remember that a refund exception was already approved. A research assistant should remember which dataset was rejected, not vaguely summarize it into “user prefers cleaner data.” A healthcare or financial assistant should not turn a precise historical statement into a soft personality trait because the memory layer wanted to look elegant. Cute demos tolerate this. Production systems do not. ...
Forms are boring until an AI agent has to fill one. Then the boring form becomes a surprisingly expensive machine. The agent reads the page, interprets the fields, finds the dropdowns, waits for the browser, loads dynamic options, decides what to click, serializes actions, and tries not to leak whatever the user typed into the wrong place. This is not intelligence in the glamorous sense. It is office work wearing a robotic costume. ...
The dangerous part is the workflow A developer asks an AI agent to inspect a repository. The agent reads a config file. Normal. It checks a failing script. Normal. It edits a helper file. Still normal. It runs a command to verify the fix. Boringly normal. Then the accumulated workflow has copied sensitive variables, modified a dependency hook, or executed a command that no one would have approved if it had appeared as a single explicit request. ...
Mathematics has a management problem. That sounds less romantic than saying it has a reasoning problem, but romance is not usually where bottlenecks hide. A proof can be brilliant, a referee can be diligent, and still the verification system can fail for the boring reason that nobody has enough time to check everything line by line. The paper Automatic Textbook Formalization takes that bottleneck seriously and then does something unusually concrete: it reports a multi-agent system that formalized a 500-plus-page graduate algebraic combinatorics textbook into Lean, with all 340 target definitions and theorems proved, in about one week.1 ...
A spreadsheet is a cruel test of artificial intelligence. Not the toy spreadsheet used in demos, with six rows, three columns, and a suspiciously cooperative universe. I mean the kind of table a real analyst asks for: every qualifying supplier in a region, every product SKU released over a decade, every regulatory filing matching a narrow condition, every competitor with exact addresses, dates, sources, and no missing cells because apparently human suffering needs columns. ...
The agent did not forget. The system outsourced remembering. Memory sounds like a solved engineering problem until an agent has to use it for work. A customer-support agent remembers the refund policy but not why an exception was approved. A research agent retrieves the right document but loses the reasoning trail that connected three earlier notes. A workflow agent crashes halfway through a task, comes back online, and must reconstruct its own state from search results like a detective investigating a crime it personally committed. ...
AI teams like committees. Not human committees, of course. Those are unfashionable. We now prefer committees made of agents: one agent plans, one verifies, one critiques, one searches, one writes code, one supervises the others, and somewhere in the corner a “coordinator” burns tokens making everyone feel aligned. This architecture is not stupid. Multi-agent systems solve real problems: they divide labor, preserve specialized expertise, and make complicated workflows easier to inspect. But they also bring the usual committee tax: coordination overhead, fragmented context, brittle phase ordering, and the faint smell of process worship. ...
A dashboard says revenue demand has “stabilized.” A monitoring agent says a sensor spike is “temporary.” A trading assistant says volatility has “fallen after the regime shift.” The sentence is smooth. The chart is nearby. The user is tired. That is usually enough for a bad explanation to survive. This is the quiet problem behind AI-assisted analytics: not whether a language model can write a plausible story about time-series data, but whether the story is faithful to the numbers. A recent paper, LLM-as-a-Judge for Time Series Explanations, studies exactly this gap by asking models to play two different roles: narrator and critic.1 ...