RAG Security

TL;DR for operators An enterprise assistant may retrieve a long internal document because one sentence is relevant to a user question. If the entire passage is then handed to the generator, the model receives much more information than it needs—and a successful prompt injection has more material available to disclose. KFS-RAG addresses that exposure point after retrieval but before generation. Instead of forwarding raw passages, it identifies query-relevant evidence and converts it into a compact set of facts. In the paper’s open-domain QA evaluation, this reduced untargeted chunk recovery from 70.8% with Vanilla RAG to 21.03%, while answer BLEU-1 and ROUGE-L remained close to the raw-context baseline. ...

TL;DR for operators Prompt-injection defense should not be procured as a leaderboard winner. A systematic review of 88 studies finds defenses distributed across several parts of the LLM system, from training and prompt handling to document boundaries, tool execution, output filtering, and continuous testing.1 Fifty-six of those 88 approaches, or 63.63%, are model-agnostic: they can operate around different models without changing model weights or architecture. That matters for teams building on proprietary APIs. ...

Customer support bots are not supposed to have enemies. They sit politely inside enterprise websites, read policy documents, retrieve relevant snippets, and answer questions with the soft confidence of a well-trained assistant. The selling point is simple: Retrieval-Augmented Generation, or RAG, should make large language models less likely to hallucinate because the answer is grounded in external evidence. ...