HotPotQA

A review meeting has one obvious purpose: prevent one person’s mistake from becoming everyone’s plan. That sounds mundane until we remember how many LLM agent systems are currently designed like a one-person review meeting. The same model attempts the task, explains why it failed, writes advice to itself, stores that advice in memory, and then tries again. It is actor, evaluator, critic, therapist, and occasionally courtroom stenographer. Efficient, yes. Also a little suspicious. ...

Every agent platform eventually develops a storage problem and pretends it is a memory strategy. The logs are all there: user turns, tool calls, partial plans, failed attempts, corrected answers, retry traces, database lookups, compliance notes, and the occasional heroic workaround that actually solved something. The tempting move is obvious. Store everything. Embed everything. Retrieve whatever looks semantically close. Then call it “long-term memory,” because “expensive junk drawer with cosine similarity” sounds less fundable. ...