RAG is a very polite librarian. It fetches documents, quotes passages, and helps an agent look less ignorant in public. Then the agent closes the book, answers the user, and leaves no trace except a chat log, a cache entry, or perhaps another small pile of private “reflections” that no one else will ever see. ...
Uncertainty is the most convenient word in governance. A model is uncertain, so the system waits. A committee is uncertain, so the decision is deferred. A risk officer is uncertain, so the memo gets another paragraph of decorative caution and nobody quite owns the next step. Very mature. Very responsible. Also, sometimes, very useful for avoiding responsibility while looking intellectually respectable. ...
A robot pauses in front of a table. There is a block, a can, a box, and something that is either on top of something else or merely enjoying a close and misleading friendship. A camera sends pixels. A perception model sends predictions. A planner wants a symbolic fact: On(A, B) or not. The expensive mistake is pretending that this last step is clean. ...
TL;DR for operators A beautiful backtest can still be a lie. Not because the model is malicious, obviously; spreadsheets have not yet formed a union. The problem is simpler and more expensive: a model can fit past data while misrepresenting the thing you actually care about. Charles Rathkopf’s paper on hallucination and reliability in scientific generative AI gives operators a useful way to think about this problem.1 It argues that hallucination should not be defined mainly as deviation from training data. In science, and in business domains that behave like science, the real question is whether an output misrepresents the target phenomenon: a protein, a weather system, a molecule, a patient, a market, a factory, a supply chain. ...