Spend the Second Write Where It Pays

TL;DR for operators

A continuously updated LLM does not necessarily need to modify its parameters for every new fact. If retrieval already makes an update answerable, the expensive second write may add little.

ALLOT, a hybrid-memory routing method from Huang, Fang, Xiao, and Du,¹ keeps every factual update in external retrieval memory and ranks updates for additional parametric adaptation. A hard cutoff then assigns the available adaptation budget to only the highest-ranked facts.

On CounterFact with Qwen3-4B, allocating parametric writes to 20% of updates produces 0.760 factual-recall accuracy, versus 0.732 under random allocation and 0.768 for a budget-matched oracle. That corresponds to 78.4% of the oracle improvement over random while using 80% fewer parametric writes than adapting every update.

For infrastructure operators, the interesting mechanism is not a particular 20% target. It is the separation of priority from budget: learn one ranking of updates, then change the cutoff as GPU or storage capacity changes. The main boundary is equally important. The evidence covers CounterFact and zsRE using Qwen3-0.6B and Qwen3-4B; it does not establish a universal adaptation percentage across domains or model families.

The expensive part of memory is the second write

Suppose an LLM product receives a continuing stream of factual updates. External memory offers a relatively cheap path: store the fact, retrieve it when relevant, and leave the base model unchanged. Parametric adaptation goes further by modifying learned parameters—in this study through LoRA adapters—but consumes training capacity and creates additional state that must be stored.

The natural response is to use both mechanisms for sufficiently important facts. The problem is that “important” does not tell an operator whether the parametric write actually improves the model once retrieval is already available.

ALLOT changes that decision. Every update is stored externally. The scarce choice is whether an individual fact also deserves parametric adaptation.

This distinction matters because the value of the second write is heterogeneous. The paper defines an item’s marginal parametric gain as the change in probe accuracy between using both memories and using external memory alone. The allocation problem is then straightforward in principle: under a budget of at most $B$ parametric writes, choose the facts with the largest achievable gains.

The difficulty is that those gains are not known for new updates.

Learn the priority once, enforce the budget separately

During supervision, the researchers evaluate labeled facts under four actions: no write, external-only, parametric-only, and both memories. They then train a class-weighted MLP using frozen text representations, retrieval-neighborhood statistics, and relation metadata.

The deployed score is not a direct prediction of marginal gain. Instead, the router sums its predicted probability that an item belongs to either action class involving parametric memory:

Updates are sorted by $s_i$, and only the top $B$ receive parametric writes.

That separation between ranking and cutoff is the method’s most useful systems property. The router does not need to be retrained when the permitted number of writes changes. A deployment with a 10% budget and one with a 30% budget can use the same ranking and move the cutoff.

It also explains why ordinary classification accuracy is not the central deployment metric. Under a hard budget, a poorly ranked update consumes a slot that could have gone to an update with greater marginal gain. Errors near the cutoff directly change which scarce writes are purchased.

At 20%, the ranking closes most of the allocatable gap

The main budget comparison uses 500 CounterFact updates with Qwen3-4B, summarized across three 80/20 partitions.

At the 20% operating point, ALLOT improves accuracy by 2.8 percentage points over random allocation and sits 0.8 points below the budget-matched oracle. Using the paper’s unrounded means, it captures 78.4% of the oracle’s improvement over random.

The result is stronger than saying that selective adaptation beats indiscriminate omission. All methods in this comparison receive the same number of parametric writes. The varying component is which facts receive them.

ALLOT leads the listed non-oracle methods from 10% through 30%. At the tightest 5% budget, balanced logistic regression is slightly ahead, 0.714 versus ALLOT’s 0.713. The reported advantage is therefore a budget frontier, not uniform domination at every cutoff.

More signals help together, not automatically

The feature study is an ablation: its purpose is to identify which information contributes to the ranking, not to establish a separate claim about retrieval quality.

At a 20% budget, text features alone recover 72.2% of oracle gain. Adding only retrieval statistics reduces that figure to 69.1%, while adding only relation metadata produces 62.9%. Combining all three reaches 78.4%.

The result warns against a simple feature-count interpretation. Retrieval context and relation metadata are useful in combination with the rest of the representation at this operating point, but neither contributes monotonically when appended individually. A direct retrieval-confidence heuristic performs much worse, recovering only 4.3% of oracle gain at 20%.

For a production router, this suggests that “retrieval confidence” should not be treated as a sufficient proxy for the value of parameter adaptation. The actual decision concerns incremental benefit after retrieval, not retrieval quality by itself.

The storage result shows why allocation can matter financially

Complementary Qwen3-0.6B shared-store experiments test a different practical question: can selective writing retain the accuracy of writing every update into both memories?

On zsRE, always writing to both memories reaches 0.644 accuracy. ALLOT reaches 0.646 while parametrically writing only 6% of updates. Estimated archived storage falls from 553 MB to 37 MB.

On CounterFact, both approaches record 0.585 accuracy, but ALLOT uses parametric writes for 14.5% of updates, with estimated archived storage of 63 MB rather than 553 MB.

These experiments are not the main Qwen3-4B budget frontier, and their absolute storage figures belong to the study’s particular adapter setup. Their stronger operational contribution is the demonstrated possibility that universal retrieval coverage and selective internalization can coexist without an observed accuracy penalty in these settings.

For continuously updated products, Cognaptus would interpret this as a governance mechanism for adaptation capacity. GPU training time and adapter storage become explicit resources that can be allocated according to predicted incremental value rather than scaling automatically with incoming update volume.

The right budget depends on the model

One result blocks an easy but misleading deployment shortcut. On CounterFact, the share of updates benefiting from both memories rather than retrieval alone rises from 19.6% with Qwen3-0.6B to 47.2% with Qwen3-4B.

A write percentage that looks adequate for one model therefore does not describe the same allocation problem for another.

The transfer experiment provides some encouragement that the ranking itself may be reusable. A router trained on zsRE reaches 0.577 accuracy on CounterFact, compared with 0.516 for retrieval-only and 0.585 for an in-domain router. The paper estimates that this preserves about 88% of the in-domain routing gain without CounterFact routing labels.

That is evidence of cross-benchmark transfer between these two factual-update datasets, not evidence of domain-independent routing. It is best read as a reason to test reused supervision before paying for a full target-domain labeling process.

Where the evidence stops

The study provides moderately strong controlled benchmark evidence, but the deployment boundary is narrow. CounterFact and zsRE contain 500 updates each; the main Qwen3-4B frontier is averaged over only three partitions; the evaluated models are Qwen3-0.6B and Qwen3-4B; and factual correctness is measured using three paraphrase probes with substring matching against the gold entity.

Router supervision is also not free. Labeled examples require measuring outcomes under alternative memory actions. The authors report that 100 labeled routing items recover roughly 55–61% of oracle gain at an estimated cost of about 3.5 GPU-hours, which makes the supervision burden measurable rather than negligible.

Sequential replay is reassuring but still short relative to a long-lived production knowledge stream: across 30 CounterFact/Qwen3-4B replays, adaptive pacing reaches 0.75764 accuracy at a 20% budget versus 0.75800 for full-pool selection.

The appropriate deployment lesson is therefore not “write 20% of facts into parameters.” It is to measure the incremental value of adaptation, learn a ranking suited to the deployment model and update stream, and treat the write budget as an operational control.

That is a more durable principle than any percentage reported in the benchmark.

Cognaptus: Automate the Present, Incubate the Future.