RAG

Search gets expensive quietly. At small scale, an embedding is just a vector. At product scale, it becomes rent: storage rent, memory rent, GPU rent, latency rent, and the recurring emotional tax of explaining why a semantic search feature needs yet another infrastructure budget. Dense embeddings made this bargain feel natural. More dimensions, more semantic capacity. More semantic capacity, better retrieval. Better retrieval, more invoices. Elegant, if one enjoys expensive inevitability. ...

Search has a small ritual. Take two embeddings, compute cosine similarity, rank the results, and move on. The ritual is fast, familiar, and usually good enough. It is also so deeply embedded in AI infrastructure that many teams treat it less like a modeling choice and more like plumbing. That is convenient. It is not always innocent. ...

RAG has a public-relations problem. It promises grounded answers, then quietly assumes that “grounded” means “retrieved from somewhere nearby.” That assumption is convenient. It is also the kind of convenience that creates compliance incidents, medical confusion, and internal knowledge assistants that cite the wrong document with absolute confidence. A retrieval-augmented system can answer from evidence and still choose the wrong evidence. It can cite something real and still fail provenance. ...

A simulation model is not a chatbot with a spreadsheet attached. That sounds obvious until a project team starts treating the LLM as if it were the entire modeling stack: the analyst, the programmer, the validator, the documentation clerk, the statistical package, and occasionally the intern blamed when the result changes on Tuesday. The convenient story is that better prompting will tame the system. Add more examples. Add a RAG. Set temperature to zero. Smile at the demo. ...

Search is supposed to make language models safer. The model does not know something, so it searches. It finds evidence, reasons over that evidence, and gives a better answer. Very civilized. Very responsible. Then the first search query goes slightly wrong. The model retrieves a relevant-looking but misleading paragraph. It builds the next reasoning step around the wrong entity. The next query becomes narrower, but in the wrong direction. The final answer may still sound fluent, because fluency is the one department where language models rarely file sick leave. The actual reasoning chain, however, has already drifted. ...

Memory is easy to sell. Give an AI agent a bigger context window. Add a vector database. Store every user preference, meeting note, support ticket, and half-correct instruction that ever passed through the system. Then call it “persistent memory,” because apparently a drawer full of old receipts is now intelligence. The problem is that agents do not fail only because they forget. They also fail because they remember too much, too flatly, and too obediently. Old facts compete with new ones. Repeated but trivial details crowd out rare but important constraints. Retrieval brings back something semantically similar but temporally wrong. The agent sounds confident because the database found something. Very helpful. Very dangerous. ...

A chatbot should not be the only employee in the company responsible for keeping secrets. That sounds obvious until we look at how many enterprise RAG systems are designed. A user asks a question. The system retrieves internal documents. The documents are placed into the model context. A policy instruction is added somewhere above the user prompt: do not reveal sensitive information. Then everyone hopes the model behaves. ...

Accuracy is comforting because it gives us a number. The model predicted the right label. The chatbot chose the same option as the survey respondent. The simulated customer picked the same product. Everyone claps, someone updates a dashboard, and the alignment problem is declared mostly solved. Unfortunately, decision-making is where accuracy goes to look respectable while quietly doing very little. ...

The demo is familiar now. Someone drops a whole contract, a whole policy manual, a whole code repository, or a month of chat history into a model and asks one neat question. The model answers fluently. The room relaxes. The slide says “1M-token context.” Procurement starts smiling. This is where the trouble begins. ...

Memory fails in a very ordinary way. A customer asks, “Can we use the same approval condition as before?” A research agent says, “Yes.” A procurement assistant retrieves the old vendor quote. A planning copilot remembers a hotel price from yesterday’s itinerary. Everything looks semantically relevant. The words match. The entities match. The embedding score smiles politely. ...