RAG’s Receipt Problem: When Correct Answers Don’t Prove Retrieval Retrieval-augmented generation has become the respectable outfit enterprise AI wears when it wants to look grounded. Add a document store, retrieve a few passages, attach citations, and the answer suddenly appears more disciplined than a free-floating chatbot. That appearance is useful. It is not proof. ...
A support ticket goes wrong. A workflow agent chooses the wrong tool. A finance assistant misses a procedural step. The usual response is familiar: add the failure to memory, rewrite a prompt, perhaps ask the agent to “reflect” before trying again. This is useful, in the same way that putting a sticky note on a broken machine is useful. It may prevent the same mistake next time. It does not prove the machine has learned how to improve. ...
A benchmark score is easy to quote. It is harder to know what broke. In Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models, Pavlos Ntais reports an 81.0% attack success rate against GPT-OSS-20B on a held-out 200-item test set.1 That number is attention-grabbing. It is also not the main lesson. ...
Opening — Why this matters now Enterprise AI is entering its less glamorous, more useful phase: not “Can we connect an LLM to everything?” but “Can we adapt it without making the GPU bill look like a small infrastructure project?” Fine-tuning still matters. Retrieval helps with knowledge access, prompt engineering helps with behavior shaping, and agent frameworks help with workflow orchestration. But many businesses eventually hit the same wall: the base model is close, yet not close enough. It needs domain style, task format, compliance habits, tool-use discipline, or workflow-specific judgment. That usually means some form of supervised fine-tuning. ...
Opening — Why this matters now The current AI market has become very good at producing fluent answers and very bad at explaining where those answers came from. This is not a minor inconvenience. It is the difference between an assistant that can be trusted in an operational workflow and an assistant that merely performs confidence with attractive typography. ...
Opening — Why this matters now Enterprise AI is entering its less glamorous phase: not the demo, not the keynote, not the charming chatbot that answers three curated questions correctly, but the operational grind of making models behave reliably inside messy workflows. That grind usually runs into a familiar triangle. Full fine-tuning is powerful but expensive, operationally heavy, and often risky when the training set is narrow. Parameter-efficient fine-tuning, especially LoRA-style adaptation, is cheaper and easier to deploy, but the smallest adapters can hit a ceiling. Meanwhile, the business user does not care whether the adapter was elegant. They care whether the model stops making the same costly mistakes in invoicing, compliance review, customer support, code generation, or scientific triage. ...
Opening — Why this matters now The AI industry has spent the last two years trying to turn large language models into workers. The result is a small circus of agents: coding agents, browser agents, research agents, support agents, spreadsheet agents, and agents that appear to exist mainly to summon other agents. Naturally, the next problem is not intelligence. It is management. ...
Cloudy With a Chance of Local Models: When On-Prem AI Starts Beating the API Server room. That phrase used to sound like a warning label in enterprise AI strategy. If a company wanted serious model capability, the usual advice was simple: use a cloud API, negotiate procurement terms, and pretend the legal team was not reading the data-processing agreement with growing despair. ...
Forecasting is where executive optimism goes to be measured. A sales team says the pipeline is healthy. A policy team says the election risk is manageable. A trading desk says the market has mostly priced in the event. Everyone has a probability. Few people have a disciplined process for updating it. That is also the problem with many AI forecasters. They can produce a number quickly, sometimes impressively, sometimes with the emotional stability of a quarterly sales forecast. But the harder question is not whether an AI can answer, “What is the probability?” The harder question is whether it can revise that probability as evidence arrives, remember why it changed its mind, and avoid turning a confidence score into decorative typography. ...
Moderation usually sits outside the model. A user sends a prompt. A model answers. Then a separate guard model steps in, reads the text, and declares the content safe or unsafe. In business terms, this is a familiar architecture: put a checkpoint at the gate, classify traffic, block what violates policy, and hope the checkpoint is both fast and sensible. It is the airport-security model of AI safety, except the passenger may be a 40-token prompt, a 4,000-token reasoning trace, or a response that is still being generated while the guard is politely looking for its shoes. ...