AI Infrastructure

Workflow diagrams age badly. A process that looked clean in January usually becomes a small archaeological site by March: one more exception, one more conditional branch, one more “temporary” manual approval that survives longer than the intern who added it. This is how many AI-agent projects quietly become ordinary software projects with a chatbot sitting on top, smiling politely while humans keep repairing the plumbing. ...

Compliance has a wonderful way of killing beautiful demos. In a demo, the agent calls a frontier model, loads a tool, reads a document, writes a decision, and everyone nods at the future. In a regulated company, the same workflow meets a less poetic checklist: where did the data go, who pays for the GPU time, can this run inside our perimeter, and why did the model spend twenty seconds “thinking” about a binary classification task? ...

A workflow looks harmless until it starts waiting on itself. One LLM call asks for a plan. Another evaluates the plan. A third revises the result. A fourth retrieves evidence. Somewhere in the middle, three subtasks could have run at the same time, two repeated calls could have been reused, and one prompt should probably have been tuned before anyone proudly called the system “agentic.” Instead, the whole thing runs as a neat little chain: expensive, slow, and quietly brittle. Very elegant, in the way a traffic jam is elegant if viewed from a drone. ...

A warehouse robot turns a corner and sees three things: a shelf edge, a moving cart, and another robot’s partial path. It does not see the blocked aisle behind the shelf. It does not see whether the cart will stop or continue. It does not see the supervisor system’s full map. Still, it must act. ...

The explanation worked in the notebook; then production happened A familiar enterprise AI story begins with a reassuring demo. A model produces a questionable prediction. Someone opens a notebook, runs SHAP, LIME, a saliency map, a concept attribution method, or whatever interpretability tool is currently fashionable enough to appear in slide decks. The plot looks plausible. The team nods. Compliance is told that explainability has been “implemented.” ...

Budget. Not the inspirational kind that appears in founder decks as “disciplined growth.” The real kind: GPU invoices, latency targets, queueing delays, memory ceilings, unhappy users, and the quiet discovery that a model can be brilliant in a benchmark and still economically annoying in production. That is the useful tension behind Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs.1 The paper does not merely repeat the familiar lesson that large language models become expensive when they get larger. Everyone with a cloud bill has already enjoyed that seminar. Its sharper point is that the usual scaling-law conversation leaves out a design variable that businesses eventually pay for: architecture. ...

Models pile up quietly. A customer-support model here. A finance QA model there. A legal drafting variant that nobody wants to delete because it passed last quarter’s evaluation. A sales assistant fine-tuned on a dataset that may or may not still represent how the company sells. Then come LoRA adapters, instruction-tuned checkpoints, safety-tuned variants, regional versions, and a few “temporary” experiments that become permanent because nobody enjoys breaking production on a Friday. ...

A prompt is usually a small thing. “White dog.” “Person in a blue jacket.” “Cup on the table.” Nobody hears these phrases and thinks: excellent, time to deploy a large general-purpose language encoder. Yet that is often what modern vision-language segmentation systems do. The visual model may be carefully optimized. The deployment team may obsess over image encoder latency, GPU memory, and batch size. Then the text side sits there, inherited from a larger foundation model stack, quietly burning capacity to understand what is often a noun phrase with a color adjective attached. Very sophisticated machinery, bravely parsing “red car.” Heroic. ...

Most office work has a draft problem. A junior analyst writes a first version of a financial memo. A lawyer marks up an argument. A consultant turns messy meeting notes into a client-ready recommendation. The first attempt is rarely useless. It is usually half-right, locally clever, and globally flawed. The expensive part is not starting from zero. The expensive part is learning how to improve a decent draft without being hypnotized by it. ...

Ask a normal enterprise RAG system a simple factual question, and it behaves politely enough. Retrieve a few passages. Hand them to the model. Generate an answer. Fine. Ask it a question that requires two or three steps, and the machine starts developing expensive habits. It retrieves, reasons, retrieves again, expands the prompt, reasons again, rewrites a query, retrieves more evidence, and then asks the LLM to stitch the mess together. The architecture looks intellectually serious. The invoice looks even more serious. ...