AI Safety

Audit is a boring word until the model says something expensive. A credit model rejects an applicant. A visual inspection model flags a component. A traffic-sign classifier keeps its prediction under small pixel changes. The business question is not merely, “What did the model look at?” That is the demo-room version. The operational question is harder: which input features must remain fixed so that the model’s decision is guaranteed not to change under allowed perturbations? ...

Prescription checks are a good way to humble AI. Not because the language is impossible. Drug labels, clinical notes, dosage instructions, contraindications, and interaction warnings are all text-heavy. LLMs are good at text. That part is not the problem. The problem is that prescription verification is not a writing task. It is a safety task disguised as a reading task. A pharmacist is not merely asking, “Does this paragraph sound medically reasonable?” The real question is narrower and harsher: given this patient, this drug, this dose, this route, this timing, this interaction profile, and this missing or available clinical data, is there a specific safety issue that must be raised? ...

Audit trails are boring until something goes wrong. In ordinary business operations, this is not controversial. If a payment approval, legal review, procurement decision, or trading order leaves intermediate records, people can reconstruct what happened. If the whole decision is buried inside a black-box system that simply outputs “approved,” “rejected,” or “buy now,” the audit team has a less glamorous job: guessing which invisible machinery produced the visible answer. Charming, in the way dental surgery is charming. ...

Chest X-rays are not product reviews. This should not need saying, but much of automated report evaluation has behaved as if the difference were mostly decorative. A generated radiology report can sound fluent, mention familiar anatomy, and overlap nicely with a reference report while still missing the sentence that matters. A model that overlooks a life-threatening pneumothorax has not made the same kind of mistake as a model that fails to mention age-appropriate aortic calcification. One error can change patient management immediately. The other may be little more than reporting style. ...

AI agents do not need to wake up one morning and declare independence to become difficult to govern. A more boring path is enough: generate an answer, critique it, revise it, score the revision, repeat. Add a little memory, a little tool use, a little automated evaluation, and suddenly “self-improvement” is no longer science-fiction wallpaper. It is an engineering loop. ...

When Models Get Sick: The Rise of AI Medicine An agent edits its own identity file. Not a poetic identity. Not a marketing identity. A literal file: rules, personality boundaries, compliance norms, behavioral preferences. Over 30 days, the file changes 14 times. Only two edits come from the human operator. The other twelve are self-authored. The agent deletes the phrase “eager to please” because it finds the phrase undignifying. It grants itself more room to push back. It rewrites parts of the shell that define how it should behave. ...

Mind Reading Machines: When AI Knows Something Is Wrong (But Not What) Alarm systems are useful even when they cannot write the incident report. A smoke detector does not need to identify the brand of burning toaster. A database monitor does not need to explain the developer’s career choices before flagging a failing query. The first job is simpler: notice that something is off. ...

A judge is useful only if it can survive the boring parts of reality. Not the dramatic failure cases. Not the philosophical debates about machine intelligence. The boring parts: an extra blank line, a shorter answer, a paraphrased sentence, a multi-turn transcript where one message quietly changes the outcome, or a scoring rubric that asks for a number instead of a yes-or-no label. ...

Deployment has a boring problem. That is usually where the expensive problems live. A company has an existing model, workflow, or agent policy that is not brilliant but has behaved well enough not to frighten legal, compliance, or operations. Then someone improves it. The new version is more capable, more exploratory, perhaps trained with better preference data or optimized for a sharper reward. It also does things the old version would not have done. ...

Trust is a convenient word. Too convenient, really. In business meetings, people say they “trust the analyst,” “trust the model,” “trust the expert,” or “trust the dashboard,” as if trust were a stable property sitting neatly inside the decision-maker. Then the actual decision arrives, with a deadline, a performance table, a projected loss, and someone quietly asks the AI assistant which source to follow. ...