Runtime Verification

The failure usually arrives after the demo A workflow agent looks excellent in a controlled demo. It reads the instruction, drafts the plan, calls the tool, produces a coherent result, and explains itself with the calm confidence of a consultant who has not yet met production data. Then the environment shifts. A document is stale. A permission boundary changes. A retrieved note is relevant but from the wrong project phase. A tool call succeeds technically while violating the user’s real constraint. A checker approves the output because the checker was never asked the right question. Nothing explodes. The system simply becomes expensive in the most boring way possible: it needs human rescue after looking competent. ...

A policy says one thing. A customer says another. A retrieved document says something newly alarming. A compliance rule says stop. A business workflow says continue. This is where large language models become interesting, and by “interesting” I mean expensive. Most companies still talk about LLM alignment as if it were a calibration problem. Tune the model. Add a system prompt. Insert a safety policy. Wrap it with retrieval. Then expect the assistant to behave consistently across messy real-world tasks. The paper Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graph argues that this expectation is too neat for the problem being solved.1 ...

The coding agent did not fail quietly. That was the point. A coding agent writes a patch. The patch looks plausible. The imports are clean enough. The function names sound like they belong in the repository. The explanation is fluent, naturally. Fluency is what these systems do best. Then the build breaks. ...