Transformers

A materials lab does not need an AI system that can politely imitate the periodic table. It needs one that can search. That difference sounds small until money enters the room. In materials discovery, every serious candidate eventually asks for simulation time, specialist review, density functional theory validation, and—if it survives long enough—lab synthesis. A model that produces many plausible crystals is useful. A model that pushes candidates toward a target property before the expensive validation begins is more useful. Less glamorous, perhaps. But so is a good spreadsheet, and civilization somehow survives. ...

Serving an LLM is usually discussed in pleasantly managerial language: latency, throughput, context windows, GPU memory, quantization, cache eviction. Nice clean nouns. Then the model ruins the spreadsheet by producing internal activations that are thousands of times larger than ordinary values, while some tokens quietly become attention magnets for reasons that are not exactly semantic. Very professional behavior from a trillion-dollar technology stack. ...

Routes look simple until someone has to pay for them. A delivery van does not care whether an optimization model sounds elegant. It cares whether the assigned route wastes fuel, crosses another vehicle’s territory, violates capacity, or produces a schedule that looks clever in a paper and stupid on the street. The Capacitated Vehicle Routing Problem, or CVRP, is where that mundane reality becomes mathematically unpleasant: multiple vehicles, limited capacity, customer demand, depot returns, and a search space that grows far faster than managerial patience. ...

Traffic control rooms rarely suffer from a shortage of numbers. Sensors count vehicles, lanes report flows, APIs stream updates, dashboards glow politely, and somewhere in the middle of all this a manager is expected to decide whether the next congestion wave is routine, dangerous, or about to become a public complaint. The naive answer is predictable: feed everything into a larger model. If one road sensor helps, fourteen must help more. If a Transformer can learn temporal patterns, give it the whole motorway and let attention perform its usual magic trick. ...

Furnace loading is not the glamorous side of artificial intelligence. No one gives a keynote about choosing which pile of titanium scrap should enter an induction furnace. Which is precisely why it is useful. The paper Enterprise Resource Planning Using Multi-type Transformers in Ferro-Titanium Industry applies a Multi-Type Transformer, or MTT, to two classic combinatorial optimization problems: the Knapsack Problem (KP) and the Job-Shop Scheduling Problem (JSP). It then pushes the method into a real manufacturing allocation case: selecting raw materials for a ferro-titanium furnace batch.1 ...

Budget meetings have a way of making AI theory suddenly less philosophical. Someone asks the simple question: “If we double the model size or the training data, how much better does the system get?” Then someone else opens a spreadsheet, adds a few curves, and everyone pretends the future has become manageable. This ritual has powered a large part of modern AI investment. Scaling laws made model development feel less like guesswork and more like engineering. ...

Reasoning Loops, Not Bigger Brains Scale is the easiest story in AI because everyone understands the shopping logic: buy more compute, add more parameters, train on more data, and watch the benchmark line move upward. It is also the story vendors enjoy telling, because nobody ever got fired for recommending a larger invoice. ...

When Attention Learns to Breathe: Sparse Transformers for Sustainable Medical AI Hospital AI does not fail only because models are inaccurate. It also fails because the input is messy, the compute budget is limited, the deployment environment is not a research lab, and the missing field in the patient record is somehow always the one the model wanted most. Elegant, really. ...

Tool calls are not tokens. Neither are paragraphs, reasoning blocks, spreadsheet edits, web searches, code executions, or the awkward little detours an agent takes before finally answering the user. Yet much of reinforcement learning for language models still behaves as if it must choose between two unsatisfying extremes. At one end, every token is treated as a tiny action. At the other, the whole answer is treated as one indivisible action. The first view is mathematically tidy and operationally noisy. The second is practical for verifiable tasks, but it compresses an entire reasoning process into one final score, which is a bit like reviewing an employee only by checking whether the office building is still standing. ...

The “important head” was never the whole story Audit. That is where many discussions about mechanistic interpretability become less romantic. It is pleasant to say that an AI model has “reasoning circuits.” It is less pleasant to ask which exact parts of the model must be preserved before a behavior survives, which parts are merely along for the ride, and which parts were called important only because our tools were too blunt to see inside them. ...