Forget Me Not: How RAG Turns Unlearning Into Precision Forgetting

A user asks to be forgotten. The recommender team opens the dashboard, sighs quietly, and faces the usual menu of unpleasant options.

Retrain the model from scratch, which is clean in theory and expensive in practice. Partition the data so only part of the system needs rebuilding, which sounds elegant until collaborative signals leak across groups like gossip at a small wedding. Or approximate the user’s influence with gradients and influence functions, which is efficient until similar users get nudged around because the model learned their tastes together.

That is the problem addressed by CRAGRU, a paper proposing a retrieval-augmented generation framework for recommendation unlearning.¹ Its central move is deceptively simple: stop treating forgetting as a parameter-editing problem, and treat it as a retrieval-control problem. The base recommender still proposes candidate items. The retrieval stage decides which user evidence is allowed into the prompt. The LLM then re-ranks or scores recommendations using only the filtered context.

In other words, CRAGRU does not ask the model to surgically forget everything it once absorbed. It asks the system to stop handing the model the evidence it should no longer use. That is not a small distinction. It is the whole article.

The trick is to move forgetting before generation

Traditional recommendation unlearning tends to operate where the system is most entangled: the trained model. Recommenders do not learn users in isolation. They learn neighbourhoods, shared preferences, item co-occurrences, latent factors, and graph structure. If one “Harry Potter fan” asks to be removed, the learned signal may also support recommendations for other users with similar tastes. Delete too aggressively and those users suffer. Delete too softly and the forgotten user is not really forgotten. Very dignified, very messy.

CRAGRU changes the control point.

The paper’s framework has three stages:

Base recommender → candidate items
        ↓
Retriever → filtered user history + profile + item metadata
        ↓
Prompt → LLM scoring / recommendation generation

The base recommender, such as BPR or LightGCN, generates a top-$K$ candidate list. CRAGRU then retrieves user-related interaction data, removes the data covered by the unlearning request, and keeps a selected subset of the remaining interactions. Those filtered interactions, candidate items, and auxiliary information are formatted into a prompt. The paper uses Llama3.1-8B as the generation model, asking it to predict scores for candidate movies in JSON format.

The important part is not that an LLM appears. LLMs now appear in recommender papers with the regularity of mushrooms after rain. The important part is where the unlearning operation sits. CRAGRU makes the deletion request operational at retrieval time:

Here, $D_u$ is the user’s interaction history and $D_u^f$ is the subset to be forgotten. Recommendations are then generated from $D_u^{filtered}$ rather than from the full user history.

That makes unlearning more like access control over evidence than full model reconstruction. For production teams, that framing is attractive because retrieval indices and prompt context are easier to update than recommender embeddings, graph parameters, or model shards. “Easier”, however, is not the same as “legally sufficient”. We will come back to that, because this is where the tempting slide-deck version of the paper starts getting ahead of itself.

The three filters are really three answers to the same operational question

Once forgetting moves into retrieval, a new problem appears: after removing the forbidden interactions, which remaining interactions should the system still show the LLM?

Showing everything is costly, noisy, and sometimes counterproductive. Showing too little damages personalisation. CRAGRU proposes three filtering strategies, each with a different view of what useful residual evidence means.

The preference-based strategy is the plainest. It groups items by category, calculates how much of the user’s history sits in each category, and samples proportionally. If 40% of a user’s retained history is science fiction and 20% is comedy, the prompt should not accidentally become a comedy-only autobiography. Reasonable. Almost suspiciously reasonable.

The diversity-aware strategy recognises that proportional sampling can still concentrate evidence in dominant categories. It treats the selection problem as a kind of resource allocation problem: given a limited number of interactions to retain, allocate retention across categories to maximise recommendation performance. In the paper, this is formulated as a dynamic programming problem over category-level retention ratios.

The attention-aware strategy is the most tailored. For each candidate item, it computes attention-based importance scores over the user’s interactions, then keeps the top-$K$ interactions most relevant to that candidate. This changes the question from “What best represents this user?” to “What history best explains whether this user might like this item?”

That difference matters. A generic user profile may preserve broad taste. A candidate-specific context may improve ranking because it supplies the LLM with the evidence most relevant to the decision at hand. The paper’s retrieval-strategy experiment supports this: attention-aware filtering achieves the strongest gains among the three strategies in the reported comparisons, including NDCG@10 improvements of 5.71% with BPR and 6.16% with LightGCN on ML-1M.

The main experiment says CRAGRU preserves utility, but not magically everywhere

The paper evaluates CRAGRU on MovieLens 100K, MovieLens 1M, and Netflix. It uses BPR and LightGCN as backbone recommenders, reserves 10% of interactions as the forgetting set, and compares against retraining, SISA, GraphEraser, RecEraser, SCIF, and IFRU. Recommendation quality is measured with HR@K and NDCG@K for $K = 5, 10, 20$.

The broad result is clear: CRAGRU is much closer to retraining than the partition-based and influence-based unlearning baselines in many settings, and it avoids the severe performance collapse visible in methods such as SISA on larger datasets.

Some concrete examples make the magnitude easier to read:

That last row is important. The paper reports strong overall utility, but CRAGRU is not the best number in every cell. In some NDCG@20 settings, IFRU performs strongly despite weaker hit-rate behaviour elsewhere. On ML-100K with LightGCN, RecEraser beats CRAGRU on HR@20. So the fair reading is not “CRAGRU wins everything”. The fair reading is: CRAGRU offers a strong utility-efficiency trade-off and usually avoids the worst collateral damage of conventional unlearning, especially where baselines degrade heavily.

This is still meaningful. In recommender systems, a method that keeps remaining-user quality near retraining while avoiding retraining cost is valuable. But it is not a universal ranking upgrade machine. Sadly, the universe remains difficult.

The speed result is the business hook hiding in plain sight

The utility results make CRAGRU credible. The efficiency results make it operationally interesting.

The paper compares unlearning time across datasets and backbones on an NVIDIA GeForce RTX 4090 GPU. CRAGRU reports 14–17 seconds across the tested conditions. The second-best baseline varies by dataset and model, but the reported improvement over the second-best method is 1.8x and 1.2x on ML-100K, 4.4x and 4.3x on ML-1M, and 7.3x and 6.9x on Netflix.

This is the most business-relevant result because deletion requests rarely arrive as one clean quarterly batch. They arrive in drips: one user, then twenty, then a regulatory audit, then a trust-and-safety workflow, then a poisoned-data removal incident that everybody hopes will not be discussed in the board deck.

Partition-based methods can look acceptable when the deletion workload is tiny and neatly localised. But recommender data is not neatly localised. The paper notes that with 10 partitions and 100 randomly selected interactions, prior work found the probability of requiring retraining across all sub-models can approach 100%. That is the operational problem CRAGRU tries to dodge by making user-level retrieval the unit of control.

The business inference is straightforward: if the model can remain stable while the retrieval layer handles many deletion requests, unlearning becomes closer to a low-latency data operation than a recurring model-maintenance fire drill. That does not eliminate governance work. It changes where the work lives.

The “completeness” evidence is useful, but needs careful interpretation

The paper’s unlearning-completeness experiment compares recommendation performance for the forgotten set versus the remaining set on ML-1M and Netflix. The likely purpose of this test is to show that CRAGRU suppresses the influence of forgotten interactions while preserving recommendation quality for users and interactions that remain in scope.

The reported pattern is consistent: recommendation quality for the forgotten set is substantially lower than for the remaining set. On ML-1M with BPR, for example, the forgotten set’s HR@1 and NDCG@1 are only 55.85% of the remaining set’s values. At higher $K$, the gap narrows: at $K=3$, the HR and NDCG ratios rise to 67.61% and 56.89%; at $K=5$, they rise to 72.36% and 59.19%.

This supports the paper’s argument that CRAGRU localises forgetting: the data that should be forgotten is less useful for recommendation after filtering, while remaining data still supports recommendations.

But the test should not be overread. Lower performance on the forgotten set is evidence that the retrieval-filtered system no longer recommends well from that removed evidence. It is not a cryptographic proof that no information about the forgotten data remains anywhere in the backbone model, logs, embeddings, item co-occurrence structure, or operational storage. The paper’s privacy analysis argues that because forgotten data is excluded from retrieval, the LLM’s recommendation depends on the filtered dataset. That is directionally sensible within the pipeline, but real production systems have more surfaces than one prompt.

This distinction is not pedantry. It is the difference between “we can prevent forgotten interactions from entering this RAG context” and “we have satisfied every interpretation of data erasure across the whole recommender stack.” Product teams may enjoy the first sentence. Legal teams will still ask about the second. As they should. That is why they are invited to meetings and then blamed for making them longer.

The ablation is really a retrieval-design lesson

The retrieval-strategy comparison is not a side quest. It explains why CRAGRU is more than “put the user history in a prompt and hope the LLM behaves”.

The paper compares CRAGRU variants against the original backbone model without retrieval filtering and against RecEraser on ML-1M and Netflix. This is best read as an ablation-style test: it asks whether the retrieval strategies themselves contribute to utility and bias mitigation.

They do. All three filtering strategies improve over the unfiltered CRAGRU baseline. Preference-based filtering preserves long-term semantic consistency; the paper reports that on Netflix with LightGCN, it improves NDCG@10 from 0.2400 to 0.2827. Diversity-aware filtering helps prevent the retained context from collapsing into narrow categories. Attention-aware filtering performs best overall because it selects interactions based on their relevance to the candidate item.

This matters for implementation. A lazy RAG recommender simply retrieves “some history”. CRAGRU’s result suggests that the retrieval policy is the unlearning policy. If retrieval is blunt, forgetting becomes blunt. If retrieval is candidate-aware, forgetting can be more precise without throwing away the rest of the user’s useful profile.

That is a useful design lesson beyond this paper. Many RAG systems treat retrieval as plumbing. In privacy-sensitive systems, retrieval is governance.

What the paper directly shows

The paper directly supports four claims.

First, CRAGRU provides a feasible architecture for recommendation unlearning that leaves the backbone recommender untouched while removing forbidden user evidence from the LLM prompt path.

Second, across three public movie-rating datasets and two recommender backbones, CRAGRU often preserves substantially more recommendation utility than partition-based and influence-based unlearning baselines. The paper reports statistically significant differences against RecEraser, with paired t-test p-values below 0.01 across datasets.

Third, the method is faster in the tested setting, with reported unlearning times of 14–17 seconds and average speedups that become larger on the bigger datasets.

Fourth, retrieval strategy matters. The filtering method is not cosmetic. It changes recommendation quality and unlearning behaviour, with attention-aware filtering giving the strongest reported gains among the variants.

Those are useful claims. They are also bounded claims.

What Cognaptus infers for business use

For businesses running recommender systems, CRAGRU points to a practical pattern: separate stable preference modelling from deletion-sensitive evidence presentation.

A production architecture inspired by this paper would not necessarily replace the existing recommender. It would layer a controlled retrieval and generation module over it:

1. the existing recommender proposes a candidate list;
2. a deletion-aware retrieval service builds the allowed user context;
3. a scoring or generation layer re-ranks the candidates;
4. deletion requests update retrieval eligibility quickly, without immediate backbone retraining.

The ROI logic is not “LLMs make recommenders magical”. Please, no. The ROI logic is that deletion requests become cheaper to process when the operational unit is user-context retrieval rather than global model surgery.

This is especially relevant in settings where recommendation quality has commercial value but deletion requests are frequent enough to make retraining painful: media platforms, marketplaces, education platforms, app stores, personalised content feeds, and internal knowledge recommenders. The more dynamic the user-history layer, the more attractive retrieval-time control becomes.

The inference is also useful for poisoned-data removal. If suspicious interactions can be filtered out of retrieval before they shape downstream recommendations, teams may respond faster while scheduling deeper model maintenance later. CRAGRU is framed around privacy unlearning, but the operational pattern also resembles rapid quarantine.

Where the result should not be stretched

The paper uses public movie-rating datasets, simulated 10% forgetting, BPR and LightGCN backbones, and Llama3.1-8B prompting. That is a serious experimental setup for a research paper. It is not the same as a production-scale advertising recommender, a healthcare recommendation engine, or a banking personalisation system under audit.

Four boundaries matter.

First, retrieval filtering is not the same as model-level deletion. CRAGRU leaves the backbone parameters intact. If the backbone itself encodes influence from the forgotten data, the paper’s architecture reduces downstream use of that data but does not necessarily remove all traces from the trained model.

Second, the evaluation focuses on recommendation metrics and simulated forgetting. It does not test membership inference, reconstruction risk, auditability, adversarial prompts, logging retention, or broader compliance requirements.

Third, the LLM prompt path introduces operational questions not fully covered by the experiments: latency under high traffic, cost per inference, deterministic output control, JSON failure handling, and privacy risks from prompt construction. The appendix shows prompt templates requiring valid JSON output, but anyone who has run LLMs in production knows “valid JSON” is less a guarantee than a polite request unless constrained decoding or validation is enforced.

Fourth, the domain is movies. MovieLens and Netflix are useful benchmarks because they are familiar and structured, but movie preferences are not all recommender behaviour. E-commerce, short-video feeds, financial product suggestions, and enterprise knowledge systems have different stakes, sparsity patterns, item lifecycles, and regulatory surfaces.

None of these limitations cancels the contribution. They simply keep the contribution in its lane. A rare and beautiful thing.

The deeper lesson: unlearning may become a systems problem, not a model problem

The most interesting part of CRAGRU is not that it uses RAG, nor that it uses an LLM, nor that it beats familiar baselines in many cells of a table. The important shift is architectural.

Recommendation unlearning has often been framed as a question of how to alter trained models after data removal. CRAGRU asks a different question: can we design the recommendation pipeline so that the deletion-sensitive layer is easier to control in the first place?

That is a more useful question for businesses. Model retraining is expensive, slow, and organisationally awkward. Retrieval control is inspectable, modular, and closer to existing data-governance workflows. It can be logged, policy-wrapped, tested, and updated without pretending the entire recommender stack is a single obedient brain waiting to be edited.

The paper does not prove that RAG-based unlearning is a complete compliance solution. It does show that, in recommendation systems, a retrieval-controlled architecture can preserve much of the utility of retraining while reducing unlearning cost and collateral damage.

That is the practical message: forgetting does not always have to begin with tearing open the model. Sometimes it begins with refusing to hand the model the wrong memory.

Cognaptus: Automate the Present, Incubate the Future.