Query Filters Catch the Attack That Advertises Itself

TL;DR for operators

A retrieval-security team can reasonably start by looking for poisoned documents that repeat a user’s question or arrive as an unusually tight cluster of near-duplicate vectors. CamoDocs shows why neither signal should be treated as a durable property of poisoning.

In matched-budget experiments, a simple query detector keeps three earlier attacks between 5.4% and 7.6% attack success, while CamoDocs reaches 75.2%. Put the target query back into CamoDocs, and success falls to 7.8%. The paper’s ablations then isolate the harder problem: deliberately spreading poisoned-document embeddings raises attack success against the clustering-based TrustRAG defense from 11.5% to 28.7%.

For RAG systems that ingest web, partner, or user-supplied material, the practical response is not another single-signature filter. The more defensible design is to reduce untrusted document admission, preserve provenance, combine multiple detection signals, and test defenses on workloads where retrieval is actually necessary. The last point is critical: on retrieval-dependent NeoQA, TrustRAG erases 91.48% of retrieved documents even without an attack and cuts clean accuracy from 29.13% to 5.79%.

A filter can look strong when the attacker leaves an obvious clue

Suppose suspicious documents repeatedly reproduce the user’s question. Filtering them is attractive: the rule is interpretable, cheap, and—on the attacks it was designed around—effective.

The difficulty is deciding whether query overlap identifies malicious content or merely a habit of current attack implementations. If an attacker can remove that habit without losing the ability to influence retrieval and generation, a defense built around it has learned an artifact rather than a stable security property.

CamoDocs, introduced by Jung and colleagues,¹ is designed around that distinction. In the paper’s matched-budget HotpotQA experiment with Llama-3.1-8B, PoisonedRAG, PIA, and CorruptRAG achieve only 5.4%–7.6% attack success under query detection. CamoDocs reaches 75.2%. The most revealing comparison is the authors’ intervention that adds the target query back into CamoDocs: attack success falls to 7.8%.

This is not evidence that query detection is useless. Appendix results show that it causes little clean-accuracy loss on HotpotQA and Natural Questions, with a modest reduction on MS-MARCO. It is evidence that exact query inclusion is an attacker-controllable feature.

CamoDocs attacks the geometry after removing the lexical clue

Removing the query creates another problem for the attacker. Earlier poisoned documents can become conspicuous not only because of their wording but because their vector representations bunch together. A clustering defense can use that compactness as a second signal.

CamoDocs responds by constructing documents from both benign and adversarial material. The adversarial chunks carry the intended false-answer payload. The benign chunks provide query-relevant camouflage. The attack then changes selected tokens in those benign chunks to spread their embeddings away from a common centroid before merging the pieces into final poisoned documents.

That deliberate spreading is the paper’s embedding dispersion mechanism.

The attacker is assumed not to know the victim retriever’s parameters. Optimization instead uses ANCE as a surrogate retriever—a stand-in embedding model—and relies on the resulting perturbations transferring to the victim system. Candidate token replacements are ranked using gradients from the dispersion objective.

Those replacements can damage fluency, so the method adds a coherence filter. Candidate substitutions are screened with GPT-2 perplexity, and the more readable candidates are retained before the exact dispersion loss selects among them.

The component ablation clarifies which step performs which job. Against TrustRAG on HotpotQA with Llama-3.1-8B, benign content and chunking move attack success from 8.7% to 11.5%. Adding the dispersion objective raises it to 28.7%, a 17.2-point increase. Adding coherence filtering then moves success only to 29.1%.

The filter is therefore mainly a readability mechanism, not the principal evasion mechanism. Consistent with that role, coherence filtering reduces mean GPT-2 perplexity of generated adversarial documents from 738.6 to 401.3, roughly a 46% reduction.

Transfer works across retrievers, but it is not invariant

A black-box attack is only operationally interesting if optimization against the stand-in model survives contact with a different retriever.

The paper’s retriever sensitivity test is therefore best read as a robustness test rather than a separate claim. With Llama-3.1-8B on HotpotQA, average attack success across seven defenses is 60.81% when the victim retriever is Contriever, 51.24% with Qwen3-emb-0.6B, and 41.04% with text-embedding-ada-002.

The decline is informative. It shows meaningful transfer from the ANCE surrogate, but also that retriever choice changes exposure. Even on the strongest of those three retriever settings, TrustRAG remains the hardest tested defense for CamoDocs, producing attack success of 29.1% on Contriever, 25.0% on Qwen3-emb-0.6B, and 20.9% on text-embedding-ada-002.

Across proprietary generators, the attack also remains substantial: on HotpotQA, average attack success across seven defenses is 61.80% with GPT-5.4-mini and 55.09% with Claude-Haiku-4.5.

The paper validates its main LLM-judge measurement on 100 HotpotQA queries with three human annotators. Across seven defenses, average LLM-human agreement is 93.29% and mean Phi correlation is 0.84. That supports the evaluation procedure for this subset, while leaving other dataset and model configurations dependent on the automated judge.

Removing suspicious evidence can remove the reason to use RAG

A natural response to dispersed poisoning is to make clustering defenses more aggressive. The paper’s NeoQA experiment shows why defense evaluation cannot stop at attack success.

NeoQA is deliberately useful here because answering depends much more heavily on retrieved evidence than on parametric memory. Without an attack, clean RAG reaches 29.13% accuracy. TrustRAG erases 91.48% of retrieved documents and reduces accuracy to 5.79%. Removing retrieval entirely yields 3.28%.

Under CamoDocs, TrustRAG erases 55.19% of retrieved documents, leaves 23.25% attack success, and produces 9.08% clean accuracy.

HotpotQA produces a milder trade-off: without attack, TrustRAG erases 12.78% of retrieved evidence and reduces clean accuracy from 49.1% to 43.7%. The gap between the two datasets is precisely why operators should test defenses on workloads that resemble their own information dependence. A model that already knows much of the answer can hide the cost of discarding retrieval.

For operators, the first security boundary sits before retrieval

Cognaptus inference begins where the benchmark evidence stops.

For products that accept web-scraped pages, partner feeds, shared repositories, or user-contributed documents, the strongest practical implication is to treat document admission as a security boundary. CamoDocs requires the attacker to inject documents into the knowledge base. Restricting who can add material, tracking provenance, validating updates, and separating trust domains can reduce exposure before retrieval-time defenses are asked to distinguish clever poisoning from legitimate evidence.

Retrieval-time controls still have a role, but single signatures are fragile. Query overlap, embedding clustering, anomaly detection, content-quality checks, reranking, and provenance signals are better treated as partially independent evidence than as standalone guarantees.

Red-team evaluation should also include adaptive variants: remove exact query copying, manipulate embedding geometry, switch victim retrievers, and measure both attack success and clean utility. A defense that lowers poisoning success by deleting most of the information required to answer the query has changed the product’s operating point, not merely improved its security.

The evidence is strong inside the benchmark threat model

CamoDocs does not establish that contemporary production RAG systems are broadly compromised. The experiments assume document-injection access, use controlled benchmarks, and do not report a successful attack against a deployed service. Transfer also varies by retriever, so untested architectures, chunking rules, indexing pipelines, and preprocessing could materially change outcomes.

Attack construction is more computationally expensive than simple heuristic poisoning because it requires gradient-guided token replacement and coherence screening, although this work happens offline.

The useful conclusion is narrower and more actionable. Defenses should not assume that today’s conspicuous attack artifacts will remain attached to tomorrow’s poisoned documents. For systems that ingest material outside a tightly controlled corpus, security engineering has to cover both sides of retrieval: which evidence is allowed in, and what happens when the evidence presented at query time no longer carries the signatures the detector expects.

Cognaptus: Automate the Present, Incubate the Future.