When One Bad Memory Hides Another

TL;DR for operators

A persistent-memory agent can retrieve two harmful records that substitute for one another. Remove either record alone, and the remaining one can preserve the same wrong answer, making both look harmless under a one-at-a-time deletion test.

MeClear¹ addresses that failure mode by measuring each retrieved memory across multiple combinations of the surrounding context rather than only against the full retrieved set. It uses sampled Shapley contributions to identify negatively contributing memories, ranks them, and then tests nested clearance candidates. A nonempty clearance is accepted only when it produces positive empirical task gain and passes a fixed recovery check.

On 745 behaviorally verified LoCoMo-derived cases, MeClear reports 85.9% Recall@|M|, 47.0% Exact Set Match, and 82.3% Binary Task Recovery. Its recovery rate is 25.5 percentage points above the Leave-One-Out baseline. The practical proposition is not “delete suspicious memories.” It is to place a reversible validation layer after retrieval and before generation, while leaving persistent storage unchanged.

The boundary is substantial: the benchmark uses controlled injected faults, top-5 frozen retrieval, and a specific model and memory stack. The formal clearance guarantee applies only to the empirical evaluator and the nested candidates actually tested, not to every possible subset or production failure distribution.

Removing one record can leave the failure untouched

Suppose an agent retrieves several memories for a query and answers incorrectly. The natural diagnostic is straightforward: remove one memory, rerun the task, and see whether performance improves.

In one representative temporal-conflict case from the paper, two injected memories both say that John was in Italy in January 2023 when the correct answer is December 2023. With both harmful memories present, task value is 0.50. Remove either one and task value remains 0.50, with no recovery. Only after both are cleared does task value return to 1.00.

That is the failure mode behind the paper’s argument. A one-at-a-time test—Leave-One-Out, or LOO—measures what changes when a record is removed from the current full context. If another harmful record can independently sustain the same error, the measured effect of deleting either record can be zero.

The paper formalizes this redundant-harm case. Two substitutable harmful memories can satisfy

The first equality says LOO sees no effect. The second says that when contribution is averaged across different memory combinations, each record still receives part of the negative utility.

Near-zero deletion effect therefore does not establish harmlessness.

MeClear evaluates a memory across different surrounding contexts

Once local deletion becomes unreliable, the attribution question changes. Instead of asking only, “What happens if I remove this memory from the full retrieved set?”, the system asks how that memory changes task utility across many subsets of the retrieved context.

MeClear treats the frozen retrieved memories as players in a cooperative game. For each memory, its Shapley contribution averages its marginal effect across possible predecessor coalitions. Exact Shapley computation is combinatorial, so the implementation samples random memory permutations and averages the observed marginal effects.

This is the paper’s central technical move. LOO remains in the method, but only as a local diagnostic. Operational harmfulness is determined by sufficiently negative sampled Shapley contribution.

The distinction is important because the contribution is task-conditioned. MeClear is not assigning a permanent quality score to a stored record. The same memory may be harmful for one query and useful for another because its contribution depends on the query, retrieved companions, task agent, and evaluator.

The paper also estimates pairwise interaction effects to diagnose substitution, complementarity, and joint interference. Those interaction scores help explain structure but do not directly determine which memories are cleared.

Attribution is followed by behavioral verification

Identifying negatively contributing memories is not the final intervention.

MeClear sorts harmful candidates from the most negative contribution upward and evaluates a nested sequence of clearance sets. It does not enumerate every possible subset. Among the candidates considered, a nonempty clearance must satisfy two conditions: empirical task value must improve, and the fixed task-recovery predicate must pass. The unchanged context remains available as a zero-gain fallback.

Among admissible candidates with maximum measured gain, the system chooses the smallest clearance set.

This turns attribution into a reversible inference-time control. Selected memories are hidden for the current query rather than deleted or rewritten in persistent storage. That separation reduces the risk of permanently removing a record merely because it conflicts with one particular task.

For production architecture, this is the paper’s most consequential design choice: retrieval can remain unchanged, while an additional validation stage evaluates the active context before generation.

The largest advantage appears where local deletion should struggle

The main evaluation contains 745 paired cases derived from 368 clean queries across ten LoCoMo conversations. The authors inject and behaviorally verify 1,115 fault records, producing 375 direct conflicts, 285 redundant conflicts, and 85 joint-interaction cases.

Across the full cohort, MeClear reports:

The redundant-conflict subset is more diagnostic than the aggregate result. LOO falls to 12.3% recall and 6.3% recovery there. MeClear reaches 83.2% recall, 49.1% Exact Set Match, and 68.4% recovery.

That pattern matches the mechanism the paper is designed to address: single-record deletion becomes least informative when harmful records substitute for one another.

MeClear is not uniformly superior on every metric. ProxySPEX records fewer extra background selections and a higher overall Exact Set Match, while ContextCite has a comparable recovery rate with reported paired confidence intervals overlapping zero. The contribution is therefore more specific than “Shapley attribution wins every comparison.” The clearest result is that coalition-aware attribution substantially improves over local LOO when interaction structure masks individual effects.

More attribution quality requires more evaluations

The sampled-permutation budget is an explicit cost-quality control.

When the budget increases from $L=4$ to $L=16$, Recall@|M| rises from 67.6% to 85.9%, Complete Set Recall from 68.1% to 86.7%, and Exact Set Match from 37.3% to 47.0%.

This is a sensitivity test rather than a second thesis. Its operational role is to show that the quality of coalition-aware attribution depends materially on how many counterfactual evaluations the system can afford.

For a deployed agent, Cognaptus would interpret $L$ as a reliability-budget parameter. Higher-stakes workflows could justify more coalition evaluations, while lower-cost applications might accept weaker attribution. The paper does not establish an economically optimal budget; it demonstrates the measurable accuracy-cost trade-off.

A post-retrieval clearance layer is the practical design proposition

The paper directly shows that reversible, query-conditioned suppression can outperform one-at-a-time deletion diagnostics on its controlled benchmark.

A business implementation would extend that result into architecture: retrieve memories normally, freeze the active context, evaluate contribution, test candidate clearances, and pass only the verified context to the task agent. Persistent storage remains intact.

That design could improve three operational properties.

First, rollback safety: a memory suppressed for one query is not destroyed for future tasks.

Second, auditability: per-memory contribution estimates, interaction diagnostics, and recovery checks provide evidence for why context was altered.

Third, failure testing: evaluation suites can include redundant and joint memory faults rather than measuring retrieval relevance alone.

These are Cognaptus inferences from the method and benchmark, not production outcomes established by the study.

The guarantee is narrower than the architecture

The evaluation is concentrated on one benchmark family, controlled injected faults, ten conversations, frozen top-5 retrieval, and a specific stack that includes Kimi-k2.6, Qwen3.6-Flash, Mem0, and Qdrant. Naturally occurring memory failures may have different interaction structures.

The formal guarantee is also conditional. MeClear establishes non-degradation and minimality only relative to its empirical evaluator and the nested candidate family it evaluates. It does not prove that the estimated harmful set equals the true harmful set, that the chosen clearance is globally optimal over all subsets, or that empirical improvement transfers unconditionally to real-world task utility.

Those limits do not invalidate the mechanism. They define what should be tested next: naturally occurring memory failures, alternative agents and judges, larger retrieved contexts, and the runtime economics of repeated coalition evaluation.

The diagnostic should match the failure structure

Persistent memory introduces a reliability problem that semantic retrieval quality alone cannot solve. A retrieved record can be relevant and still reduce task performance, and several harmful records can interact so that none looks damaging when tested individually.

MeClear’s contribution is to make that interaction structure operational: evaluate memories across coalitions, treat negative contribution as query-specific, verify the proposed intervention behaviorally, and suppress context without rewriting storage.

For operators, the key change is procedural. When an agent fails after retrieval, the question is no longer only which individual memory looks suspicious. It is whether the diagnostic can detect harm that exists only in combination—and whether the intervention can be reversed after the query ends.

Cognaptus: Automate the Present, Incubate the Future.