AI Operations

Reasoning is expensive mostly because we make the model say it. That sounds almost too simple, which is usually where trouble begins. Chain-of-thought reasoning improved language-model performance by giving the model a written workspace: first solve, then answer. But the same trick also turns internal computation into external communication. Every intermediate step must be decoded, formatted, and passed forward one token at a time. The model is not just thinking; it is producing a small essay it may not need to show anyone. ...

Adapters are supposed to make fine-tuning simple. A team takes a large pretrained model, freezes most of it, trains a small adapter for customer support, another for invoice extraction, another for compliance review, and so on. The pitch is attractive: less storage, less training cost, faster iteration, fewer excuses from the infrastructure team. Naturally, the adapter becomes the small and tidy object everyone wants to manage. ...

Opening — Why this matters now Enterprise AI is entering its less glamorous, more useful phase: not “Can we connect an LLM to everything?” but “Can we adapt it without making the GPU bill look like a small infrastructure project?” Fine-tuning still matters. Retrieval helps with knowledge access, prompt engineering helps with behavior shaping, and agent frameworks help with workflow orchestration. But many businesses eventually hit the same wall: the base model is close, yet not close enough. It needs domain style, task format, compliance habits, tool-use discipline, or workflow-specific judgment. That usually means some form of supervised fine-tuning. ...

Opening — Why this matters now For the last few years, AI strategy has been narrated as a model-quality story: bigger models, better benchmarks, longer context windows, more agents, more demos, more adjectives. That story was useful. It was also incomplete. The less glamorous reality is now arriving with the invoice attached. LLM systems are not merely models. They are production services that consume GPU memory, scheduling capacity, engineering attention, and operational patience. Once a business moves from a prototype to repeated daily use, the question changes from “Can the model answer?” to “Can the system answer reliably, cheaply, and repeatedly when real users arrive at inconvenient times?” ...

Opening — Why this matters now AI agents are leaving the demo cage. They are no longer just politely completing prompts; they are planning workflows, calling tools, reading files, coordinating intermediate steps, and accumulating context like a bureaucrat hoarding PDFs. This is useful. It is also expensive. The paper “QuantClaw: Precision Where It Matters for OpenClaw” studies a problem that sounds technical but is really managerial: agent systems often run every task at a fixed numerical precision, even though not every task deserves the same computational budget.1 A safety-critical terminal command and a lightweight retrieval summary are not the same species of work. Treating them identically is the infrastructure equivalent of sending a limousine to deliver printer paper. ...

Routes look clean on a dashboard. A line leaves the depot, touches a sequence of customers, maybe bends toward a charging station, and returns home. The illusion is that route planning is still mostly about drawing the shortest useful line. Electric fleets ruin that illusion rather quickly. A diesel truck can treat refueling as an annoying but usually minor detail. An electric vehicle cannot. Battery capacity turns distance into feasibility. Charging stations turn geography into detours. A route that looks efficient before charging may become expensive after charging; a route that looks wasteful may avoid a much uglier charging pattern. This is why the Electric Capacitated Vehicle Routing Problem, or E-CVRP, is not merely the old vehicle-routing problem wearing a green jacket. It is a coupled routing-and-energy problem, and coupling is where algorithms go to lose their innocence. ...

The failure usually arrives after the demo A workflow agent looks excellent in a controlled demo. It reads the instruction, drafts the plan, calls the tool, produces a coherent result, and explains itself with the calm confidence of a consultant who has not yet met production data. Then the environment shifts. A document is stale. A permission boundary changes. A retrieved note is relevant but from the wrong project phase. A tool call succeeds technically while violating the user’s real constraint. A checker approves the output because the checker was never asked the right question. Nothing explodes. The system simply becomes expensive in the most boring way possible: it needs human rescue after looking competent. ...

Prompts are now office furniture. Everyone has them. Everyone complains about them. Nobody is quite sure who owns the standard version. One team keeps a Notion page of “best prompts.” Another hides theirs in a spreadsheet. A third tells new staff to “just ask clearly,” which is not a method, but it does have the administrative elegance of doing nothing. ...

A chatbot does not need a philosophy seminar to answer “Who directed Oppenheimer?” That sentence sounds obvious. Yet a large part of today’s AI infrastructure behaves as if every user query deserves a carefully staged internal drama: retrieve facts, reason through them, verify the logic, produce a chain of intermediate steps, and finally deliver the answer the system could have produced with a simple lookup. It is impressive in the same way using a crane to move a coffee cup is impressive. Technically capable. Operationally absurd. ...

Write-Back to the Future: When Your RAG Starts Learning A RAG system usually fails in a very ordinary way. The retriever finds something relevant, but not quite enough. The generator receives five passages, three of which are useful, one of which is decorative furniture, and one of which looks relevant only because it shares the right vocabulary. The answer is then expected to emerge from this little committee of half-helpful paragraphs. Sometimes it does. Sometimes it does what committees do. ...