Memory With a Pulse: Real-Time Feedback Loops for RAG Systems

Ask an enterprise chatbot the wrong question on the wrong day and the problem is rarely that the language model has forgotten how to write English. The problem is that it has been handed the wrong pile of evidence.

That is the expensive little defect inside many retrieval-augmented generation systems. The model may be fluent. The corpus may be current. The vector database may be humming along like a well-funded filing cabinet. Yet the answer still disappoints because the system chose the wrong snippets, placed a useful document too low, missed a newly relevant runbook, or treated yesterday’s user intent as if it were carved into basalt.

The paper behind this article, DMA: Online RAG Alignment with Human Feedback, attacks exactly that layer of the stack.¹ Its contribution is not another plea for longer context windows, larger embedding models, or more heroic prompt templates. The authors introduce Dynamic Memory Alignment, or DMA: an online RAG framework that uses document-level, list-level, and response-level human feedback to update retrieval and reranking behaviour in production-like settings.

The phrase “memory alignment” is easy to misread. DMA does not primarily align the LLM itself. It aligns the retrieval layer: the mechanism deciding what evidence enters the model’s working memory. In the paper’s terminology, “memory” is not the whole archive. It is the token-bounded context visible to the generator at a given turn. In business language: DMA does not retrain the employee. It improves the briefing pack.

That distinction matters. Most organisations do not fail at RAG because their model cannot produce a polished paragraph. They fail because their assistants keep bringing the wrong documents to the meeting.

The real object being controlled is working memory, not the corpus

A standard RAG pipeline has a familiar shape. A user asks a question. A dense retriever fetches candidate documents from an external corpus. A reranker selects and orders the top contexts. A generator produces an answer from that selected evidence.

The corporate fantasy is that this pipeline is modular, stable, and mostly solved. Train the retriever, index the documents, bolt on a reranker, then let the LLM do its soft-focus magic. The real system is less serene. User intent shifts. Product releases change the meaning of old queries. Support teams learn that some documents are technically relevant but operationally useless. New failure modes appear in logs before they appear in evaluation sets. Static retrieval becomes a tax on every downstream answer.

DMA reframes the problem as control over working memory. The corpus may contain everything. The prompt cannot. The system must decide, repeatedly and under latency constraints, which few pieces of evidence deserve to be shown to the generator.

That is the mechanism-first insight of the paper. Better RAG is not only about finding more documents. It is about learning how humans want evidence selected and ordered over time.

DMA turns messy feedback into three retrieval signals

The paper’s framework begins with a feedback taxonomy. This sounds bureaucratic, which is probably why it is useful. Feedback in deployed systems is not one clean scalar reward. It arrives as clicks, votes, regeneration requests, comparison judgements, session outcomes, and implied irritation. DMA sorts these signals into granularities that can train different retrieval-side components.

This taxonomy is more than neat labelling. It solves an interface problem. Human feedback is generated after users experience answers, but RAG quality depends heavily on what was retrieved before the answer was generated. DMA maps feedback backward into retrieval control.

Document-level feedback trains a pointwise scorer. List-level feedback pre-trains a listwise model using a ListNet-style objective. Response-level comparisons train a reward model over document lists, using a Bradley–Terry preference formulation. Then a stochastic list policy is aligned with that reward model using PPO over a Plackett–Luce ranking policy.

That may sound like a lot of machinery just to sort documents. It is. Welcome to production AI, where “just retrieve the relevant context” eventually becomes a small civilisation.

The generator stays fixed while the evidence selector learns

One of the paper’s most important design choices is what it does not update. DMA does not require the LLM generator to be retrained. In the online experiment, responses are generated by a fixed Qwen2-72B decoder. In the offline evaluation, the reader is fixed to LLaMA2-7B. The moving part is the retrieval and reranking pipeline.

That makes the framework more plausible for enterprise use. Updating a generator is expensive, risky, slow to validate, and usually entangled with governance. Updating a reranker is comparatively contained. The business can improve answer quality by improving the evidence supply chain rather than reopening the entire model stack.

DMA’s training-to-serving pathway has three main steps:

1. use feedback to train retrieval-side teacher models;
2. align a listwise policy with response-level rewards;
3. distil those teacher signals into a lightweight gradient-boosted decision tree scorer for online serving.

The final step is not a minor implementation detail. It is what makes the framework deployable. The paper reports sub-10 millisecond median end-to-end scoring latency per list for the GBDT student. Teacher models can be richer and slower; the serving ranker must be boringly fast. There is a moral here for enterprise AI architecture: the clever part may live nearline, but the production path still needs to behave like production software, not a graduate seminar with a GPU budget.

The strongest evidence is online, not just benchmark theatre

The headline result comes from a multi-month randomized controlled trial on a GenAI assistant operated by a major telecommunications/cloud provider. Sessions were split at the session level. The control used a strong static BGE-Reranker. The treatment used DMA with nearline updates. Retrieval used BGE-m3, and responses were produced by a fixed Qwen2-72B decoder.

The traffic was not a toy domain. The paper categorises queries across seven application areas: technical support at 37%, performance and monitoring at 21%, API and developer support at 16%, security and compliance at 10%, service and resource management at 9%, migration and deployment at 4%, and product features or updates at 3%. In other words, the system was tested in the kind of technically specialised environment where bad retrieval is not merely annoying. It wastes expert time.

Because explicit ratings were sparse, the authors used Qwen2-72B as a session-level satisfaction annotator, applying a calibrated few-shot prompt to the whole session trace. The reported satisfaction metric is the share of sessions labelled not dissatisfied. The paper reports high agreement with human annotations on a held-out set, with Cohen’s $\kappa = 0.962$.

That label design deserves both credit and restraint. It is better than pretending sparse thumbs-up ratings capture the whole user experience. It is also not the same as direct human satisfaction measurement at full scale. The evaluator is another model. A careful reader should treat the result as strong evidence for deployment improvement under this evaluation protocol, not as a universal law of human happiness. Humanity remains, regrettably, not fully reducible to a Qwen prompt.

The RCT result is large, but the ablations are more informative

DMA improves session-level satisfaction from 62.11% for the static BGE-Reranker to 77.37% for the full DMA system. That is a gain of 15.26 percentage points, or 24.57% relative improvement. The confidence intervals are tight: [61.64, 62.58] for the control and [76.99, 77.75] for full DMA.

The size is notable. The more interesting part is how the paper decomposes it.

The hierarchy is telling: list-level feedback matters most, followed by response-level feedback, then document-level feedback.

That runs against a common instinct in retrieval systems. Engineers often obsess over whether individual chunks are relevant. DMA’s results suggest that, at least in this setting, the bundle matters more. Users experience an answer generated from a set of contexts, not a labelled passage floating alone in vector space. A document may be relevant but redundant, relevant but badly positioned, or relevant but insufficient without a companion document. List-level feedback captures some of that structure.

Response-level feedback is the next major contributor. This makes sense because the business objective is not “rank passages elegantly”; it is “produce an answer that resolves the user’s issue.” DMA attempts to attribute response preference back to the evidence list by fixing the generator, prompt, temperature, and decoding seed across comparisons, and by using cross-generation on a stratified subset to reduce order effects. This is not perfect causal magic, but it is a serious attempt to avoid blaming retrieval for generation randomness.

Document-level feedback still helps. Removing it costs 4.08 points. But the smaller drop implies that snippet-level usefulness is a partial view of the problem. A RAG assistant is not a search engine with a chat bubble glued on. It is a context assembly system.

Distillation is where the research becomes operational

The fusion result is easy to skim past and should not be. DMA’s GBDT student reaches 77.34% satisfaction, compared with 72.79% for cascading fusion. That is a 4.55-point advantage for the distillation strategy.

Cascading fusion is the obvious engineering shortcut: combine teacher scores, interpolate them, rerank, and hope nobody looks too closely. DMA’s distillation compresses teacher logits and retrieval features into a single production scorer. The paper reports that this approach also improves tail-latency adherence and lowers variance across traffic slices.

This matters because enterprise systems rarely die from one bad benchmark score. They die from operational friction: unstable latency, fragile rollouts, incompatible feature schemas, and weekly model refreshes that turn into quiet governance theatre. DMA includes feature gating, schema versioning, shadow evaluation, and incremental rollout safeguards. These are not glamorous contributions, but they are the difference between “research result” and “system one might actually be allowed to deploy.”

The nearline update cadence is also important, though the effect is more modest. Switching from weekly batch refreshes to nearline online learning improves satisfaction by 1.33 points, from 76.21% to 77.54%. The paper notes larger drops during distribution shifts such as product release weeks. That is precisely where enterprise support systems need adaptation: not when yesterday looks like today, but when some product team has shipped “a small update” and the support queue has become a live archaeological site.

Offline benchmarks confirm transfer, but not universal dominance

The paper also evaluates DMA on public QA benchmarks using a fixed LLaMA2-7B reader. This is best understood as a robustness and transfer test, not the main proof of business value. The core claim is online adaptation under deployment constraints; static QA benchmarks can only partially test that.

Still, the pattern is useful. DMA achieves the best reported Hit@1 and F1 on the conversational QA datasets in the table: TriviaQA and HotpotQA. On TriviaQA, DMA reports 68.81 Hit@1 and 68.90 F1, above FILCO’s 67.30 and 67.80. On HotpotQA, DMA reports 33.92 Hit@1 and 41.88 F1, above FILCO’s 32.70 and 40.80.

On schema-bound datasets, the story is less triumphant. DMA is competitive on Natural Questions, with 51.11 Hit@1 and 54.92 F1, but it does not beat FILCO’s 52.71 Hit@1 and 55.32 F1. On WebQSP, DMA’s 67.26 Hit@1 and 65.03 F1 trails FILCO’s 69.96 and 68.34.

That pattern supports the paper’s own framing. DMA appears strongest where conversational, multi-evidence grounding matters. It is less clearly superior on more entity-centric or schema-bound tasks. This is not a failure. It is a boundary. Systems built around live user intent and evolving support content are closer to DMA’s home territory than static lookup tasks.

The business value is retrieval governance, not model glamour

For companies building internal knowledge assistants, customer support agents, developer copilots, or technical operations bots, DMA points to a more practical optimisation target.

The usual executive question is: “Which model should we use?” The DMA answer is, effectively: “After a point, the model is not the bottleneck you think it is.” If a fixed generator improves substantially when the retrieval layer adapts, then some budget should move from model shopping to evidence governance.

This suggests three business implications.

First, feedback must be collected at the level where decisions are made. A thumbs-up on the final answer is useful but insufficient. Good RAG operations need instrumentation around snippets, retrieved lists, regenerated answers, response comparisons, and session outcomes. Without that, feedback remains a dashboard decoration. Lovely colours, limited intelligence.

Second, retrieval should be treated as an adaptive policy, not a static utility function. The reranker should learn from what users actually resolve, ignore, regenerate, or prefer. This is especially relevant in domains with fast-changing product documentation, compliance updates, incident playbooks, or customer-specific procedures.

Third, production constraints should be designed into the learning loop from the start. DMA’s GBDT distillation, nearline update trigger, shadow evaluation, and rollout controls are not afterthoughts. They are the operational skeleton. A system that improves quality but destroys latency is not aligned with business reality. It is merely ambitious in the wrong dimension.

What the paper directly shows, and what Cognaptus infers

The paper directly shows that, in one large technical deployment, DMA improved model-inferred session satisfaction over a strong static BGE-Reranker by 15.26 percentage points. It also shows that removing list-level, response-level, and document-level feedback reduces satisfaction, with list-level feedback producing the largest drop. It shows that GBDT distillation outperformed cascading fusion, and that nearline learning outperformed weekly refreshes. Offline, it shows stronger results on conversational QA benchmarks than on schema-bound ones.

Cognaptus infers that the framework is most relevant for high-volume enterprise RAG systems where three conditions hold: feedback is abundant enough to support repeated updates, user intent is non-stationary, and the cost of wrong context is material. Customer support, cloud operations, developer documentation, internal IT helpdesks, compliance advisory, and field-service knowledge systems fit that profile better than low-volume executive Q&A bots.

What remains uncertain is portability. The main online evidence comes from a technically specialised telecom/cloud assistant. The satisfaction labels are largely model-inferred, albeit with strong reported agreement against human annotations. The paper does not prove that DMA will deliver the same magnitude in consumer chat, legal research, healthcare triage, finance advisory, or low-feedback environments. Nor does it solve feedback about answer style, reasoning quality, or tone, because the generator remains fixed.

The sensible business reading is therefore not “DMA is the universal future of RAG.” That would be the usual brochure-grade overreach. The better reading is: retrieval alignment deserves a production feedback loop, and DMA offers a serious blueprint for building one.

The boundaries are as important as the mechanism

DMA aligns the retrieval layer. That is its strength and its limit.

If a user dislikes an answer because it is too terse, too verbose, legally unsafe, badly reasoned, or written with the emotional warmth of a printer manual, DMA may not fix the core issue. Better evidence can improve factual grounding, but it cannot guarantee better reasoning or communication style. The paper explicitly notes that feedback related to answer style or reasoning quality may require cross-layer alignment beyond retrieval.

The reward model is also list-level, which means it can miss fine-grained factual signals. A response may be preferred for reasons that are not cleanly attributable to the retrieved list. The authors attempt to control decoding confounds by fixing generation settings during comparisons, but attribution remains an inherently difficult problem.

Another boundary is retriever stability. DMA assumes a stable retriever backbone. Major retriever updates can cause embedding drift, weakening alignment transfer. This is a practical concern. Enterprises love upgrading components right up to the moment their carefully calibrated ranking behaviour quietly changes.

Finally, nearline updates still require infrastructure. The paper’s operational setup triggers updates after roughly 500 confidence-filtered feedback instances, refreshes teachers, runs PPO alignment, and distils into a 10K-tree GBDT student. The reported update cycle averages about 10 minutes on 8 A800 GPUs. That is manageable for large deployments. It is not free.

Memory with a pulse is a better metaphor than memory with a warehouse

The old metaphor for RAG was storage. Put knowledge somewhere the model can retrieve it. Keep it fresh. Search it well. This was useful, but incomplete.

DMA pushes toward a better metaphor: working memory as a live allocation problem. The system does not merely retrieve from a warehouse. It learns which evidence deserves attention now, for this user intent, in this domain state, under this latency budget.

That is where the paper earns its keep. The contribution is not that human feedback matters. Everyone says that. The contribution is a concrete mechanism for turning heterogeneous feedback into retrieval-side learning, validating it in an online RCT, and compressing the result into a production scorer.

For enterprise AI, the message is pleasantly unsentimental. Bigger models may help. Longer context may help. Better prompts may help. But if the system keeps handing the model the wrong evidence, then all that intelligence is being asked to improvise from a bad briefing.

DMA suggests a more disciplined path: make the memory layer adaptive. Let feedback change what the model sees. Keep the generator stable when that is operationally safer. Distil the cleverness into something fast enough to serve. Measure the whole session, not just the prettiness of the answer.

In other words, stop treating RAG memory like a filing cabinet. Give it a pulse.

Cognaptus: Automate the Present, Incubate the Future.