Opening — Why this matters now Agentic AI systems are currently being sold with a suspiciously comforting ritual: generate an answer, ask the same model to reflect, then ask it to improve the answer. Repeat until the dashboard looks busy. In demos, this feels intelligent. In production, it may simply be a very expensive way to turn correct answers into wrong ones. ...
A thermostat can be annoying in a very ordinary way. It does not need to fail dramatically. It only needs to keep switching equipment on and off, chasing tiny temperature deviations as if every small fluctuation were a crisis. The room stays mostly comfortable. The dashboard may even show acceptable performance. But behind the polite control signal, compressors cycle, dampers move, energy bills creep upward, and maintenance teams inherit the consequences. ...
Factory robots, drones, and autonomous vehicles do not usually fail because nobody cared about safety. They fail because “safe” is annoyingly difficult to write down. An operator may know that a drone should not scrape the ground, that a warehouse robot should not cut across a human worker’s path, or that an autonomous car should not tailgate even when the road is technically clear. But turning that judgement into a formal mathematical boundary is another matter. The physical system has dynamics. The controller has limits. The dangerous state may not be a simple wall or circle. And the difference between “safe enough” and “please do not put that in production” may live in patterns of behaviour rather than in a clean rule. ...