After the Bad Memory: Repairing the Decisions It Already Touched

TL;DR for operators

When an agent discovers that a stored customer preference, prior observation, or workflow fact was wrong, deleting that record may be too late. The faulty information may already have shaped a plan, triggered a tool call, entered the final answer, or created new persistent memories.

Yu et al. propose a repair mechanism that follows those dependencies rather than resetting everything.¹ On their 150-case controlled benchmark, it recovered 85.3% of cases, compared with 77.3% for LLM-judge repair, while reducing the replay ratio from 21.7% to 12.3% and average LLM calls from 9.80 to 5.70.

The result is not a claim that selective repair dominates every alternative. The method had worse recurrence than LLM-judge repair on the controlled benchmark and substantially lower claim-invalidation F1 than two trace-centric approaches. Its strongest practical contribution is narrower: once a memory fault is known, explicit dependency information can help decide what state to remove, what valid state to retain, and what computation actually needs to run again.

For operators, this moves memory incident response upstream into system architecture. A recoverable agent needs provenance for memory reads, claims, actions, observations, and memory mutations. It also needs safe ways to repeat or compensate for tool actions when recovery crosses into the external world.

A bad memory can survive its own deletion

Consider an agent that stores a customer’s travel preference incorrectly. The record is later used to choose an itinerary, which leads to a booking-related tool action, which contributes to a response and perhaps a summarized memory of the interaction.

If the original record is subsequently identified as faulty, removing it fixes only one element of the system state. The itinerary decision still exists. The response may still depend on it. A derived summary may preserve the same error in a different form.

The paper formalizes this as propagation through explicit dependencies among memory records, execution steps, observations, answers, and memory mutations. The initial repair problem is therefore not “which memory should be deleted?” but “which downstream state has lost its justification because that memory was wrong?”

That change in recovery target is the paper’s main conceptual contribution.

Repair follows dependencies, but reachability is not enough

The proposed system first constructs a heterogeneous dependency graph from runtime provenance. Diagnosed faulty memories become starting points, and the system traces forward across propagation relations to identify potentially affected nodes.

This tracing is deliberately broad. A node downstream from bad information is a candidate for repair, not automatically invalid.

The next step checks whether a reachable item still has sufficient trusted support from outside both the diagnosed faults and the raw affected region. If so, it can be preserved. This independent-support check handles an important case: a claim may have been touched by faulty memory while also being fully justified by unaffected evidence.

That distinction prevents dependency tracing from becoming indiscriminate deletion.

After support checking, a deterministic rollback planner removes diagnosed faulty memories, quarantines unsupported affected memories, invalidates unsupported execution state, and isolates the portion of affected computation that can still influence the corrected final answer. Only the executable closure required for that region is replayed, with safe preserved context reused rather than recomputed.

The method is therefore selective in two different senses: it distinguishes affected from unaffected state, then distinguishes affected computation that still matters to the answer from affected computation that can remain invalidated.

The main benchmark favors recovery, not perfect trace reconstruction

The controlled evaluation contains 150 tool-use cases across shopping, travel, and customer support. Faults include poisoned memories, stale information, wrong-user records, and summary drift. The cases are deliberately constructed so that the injected fault changes the final answer and produces a downstream effect.

The main comparison gives dependency-guided rollback the highest immediate recovery:

This combination is more informative than the recovery score alone. The method appears optimized for restoring a usable answer and persistent state without replaying too much work. It is not uniformly better at reconstructing which individual claims should have been invalidated.

That matters for deployment choices. A system whose priority is immediate service recovery may value a different metric mix from one whose primary requirement is forensic trace reconstruction.

The ablations reveal three different jobs

The component ablations are especially useful because they separate the purposes of the repair stages.

Removing the rollback planner reduces recovery from 0.853 to 0.713 and raises recurrence from 0.266 to 0.430. This is the clearest evidence that rule-guided state decisions are central to the method’s immediate recovery performance.

Removing independent-support checking has a different effect. Recovery actually rises slightly to 0.880, but benign preservation falls to 0.986, replay expands, and LLM calls increase. Support checking is therefore better understood as a preservation and selectivity mechanism than as the main driver of raw recovery.

Selective replay is primarily a cost-control mechanism. Without it, the replay ratio jumps from 0.123 to 0.755 and average LLM calls from 5.70 to 24.01. Yet recurrence falls sharply from 0.266 to 0.071.

This is a genuine design trade-off. Broader replay consumes substantially more computation but appears to reduce recurrence in this experiment. Selective replay cannot be interpreted as a free efficiency gain.

Transfer evidence lowers the expected ceiling

The paper also adapts 50 procedural and navigation-style trajectories from LongMemEval-V2 into the same repair schema. This is a robustness and transfer test, not a representative evaluation of the complete LongMemEval-V2 benchmark.

Recovery falls from 0.853 in the controlled setting to 0.680 in the adapted subset. The proposed method still exceeds AgentTrace-style repair at 0.540 and LLM-judge repair at 0.260, and its claim-invalidation F1 rises to the best reported value in that subset, 0.669.

The lower absolute recovery is useful evidence. It argues against treating the controlled benchmark result as a deployment estimate. The method transfers, but the more externally derived setting is materially harder.

Recoverability becomes an architecture requirement

The paper directly demonstrates benchmark recovery behavior. The next step is a Cognaptus inference about system design.

For agent platforms that retain customer preferences, case histories, prior observations, or workflow state, dependency-aware recovery could reduce two costly responses to memory incidents: wiping valid personalization and rerunning entire trajectories.

But this capability must be designed into the runtime. The repair mechanism assumes explicit provenance connecting memory reads to claims, plans, tool actions, observations, answers, and later memory changes. Missing edges can leave contamination untraced; spurious edges can invalidate too much.

Tool interfaces create a second architectural boundary. Rolling back internal agent state does not cancel an airline booking, reverse a purchase, or undo an account modification. Systems performing consequential actions therefore need idempotent operations, resettable environments, or compensating transactions if replay is expected to be safe.

The operational evaluation should also remain multi-objective. Recovery rate alone hides whether a method preserves benign memory, leaves recurrent contamination, invalidates the right claims, or achieves its result by rerunning most of the trace.

The unresolved problem begins before rollback

The paper assumes the faulty memories have already been diagnosed. It does not test how reliably a production system can identify those faults in the first place.

That is a substantial boundary because rollback quality depends on correct starting points and accurate provenance. The controlled benchmark also uses injected faults selected specifically because they alter the answer and create downstream consequences, and the main results come from single deterministic runs. Neither setting estimates the natural prevalence or difficulty of memory incidents in deployed systems.

The contribution is therefore best read as a recovery architecture for an already-diagnosed incident, not a complete memory-safety system.

Its strongest idea is also the simplest to carry forward: persistent memory turns errors into state-management problems. Once a faulty memory has influenced execution, repairing the record and repairing the agent are no longer equivalent operations.

Cognaptus: Automate the Present, Incubate the Future.