Retrieval-Augmented Generation

Ask a normal enterprise RAG system a simple factual question, and it behaves politely enough. Retrieve a few passages. Hand them to the model. Generate an answer. Fine. Ask it a question that requires two or three steps, and the machine starts developing expensive habits. It retrieves, reasons, retrieves again, expands the prompt, reasons again, rewrites a query, retrieves more evidence, and then asks the LLM to stitch the mess together. The architecture looks intellectually serious. The invoice looks even more serious. ...

Retrieval has a breathing problem. Most enterprise RAG systems inhale once, grab the nearest chunks, and then hope the model can make the answer sound less fragile than the evidence actually is. That works tolerably well when the user asks for something sitting neatly inside a document paragraph. It works less well when the answer lives across entities, relations, aliases, product categories, authors, diseases, suppliers, regulations, or customer records. In other words, it works less well in the part of business where knowledge is not a pile of text but a network. ...

Retrieval has a management problem. Not the motivational-poster kind of management problem. The operational kind. A company asks its AI system a question about a contract, a customer dispute, a policy exception, or a technical incident. The answer is not sitting in one paragraph. It is distributed across definitions, transactions, policies, exceptions, and historical context. A flat vector search grabs a few semantically similar chunks and hopes the model can stitch them together. A global summarizer reads widely, compresses aggressively, and occasionally smooths away the exact fact that mattered. A local graph search follows nearby entities and may become very confident inside the wrong neighborhood. ...

A developer can understand what a software function should do, write something that looks reasonable, and still fail because the surrounding codebase expects a particular interface, naming convention, object hierarchy, or sequence of calls. Giving the developer four independent attempts may eventually fix a misplaced bracket. It does little when the real problem is that they do not know which internal abstraction the system expects. ...

Deal flow is rarely scarce. Attention is. A venture-capital team may receive hundreds of startup introductions, each surrounded by founder biographies, investor histories, comparable companies, co-investment relationships, sector narratives, and enthusiastic claims about an inevitable Series A. The practical problem is not obtaining more evidence. It is deciding which fragments deserve serious attention before the partnership meeting begins. ...

The inbox problem hiding inside RAG Inbox. That is the easiest way to understand what goes wrong in many retrieval-augmented generation systems. A query arrives. The system retrieves a few documents. The answer is not obvious. So the system retrieves more. Then more. Then perhaps a web search result. Then a rewritten query. Then another bundle of passages. ...

TL;DR for operators Private examples are not harmless just because they sit inside a prompt rather than inside model weights. In-context learning lets teams adapt a general LLM by adding examples at inference time, which is convenient until those examples are medical notes, legal clauses, customer tickets, invoices, or internal decisions that should not be inferable from the model’s output. ...

Hospital search is rarely a search problem in the clean, consumer-internet sense. The useful information is not sitting in one tidy index, wearing a name badge, waiting to be embedded. It is scattered across clinical notes, relational databases, knowledge graphs, departmental systems, hospital networks, and legal boundaries. Naturally, this is where people decide to add a large language model and call it “modernisation.” Brave. ...

TL;DR for operators The useful lesson from this paper is not “RAG is bad”. That would be lazy, which is traditionally how bad AI strategy gets promoted to a roadmap. The sharper lesson is this: retrieval helps when the model actually needs external knowledge, the source is useful, and the retrieved context does not interfere with the model’s own competence. In the paper’s mixture-of-knowledge setting, those conditions are not reliably true. ...

TL;DR for operators A financial AI system does not fail only when it invents a company, misreads a filing, or forgets what EBITDA means. Those are the obvious failures. FinanceBench is more useful because it exposes the quieter failure mode: the model has access to the document, produces a coherent answer, and still gets the financial question wrong.1 ...