Hallucination

Empathy is easy to fake for one sentence. A chatbot can say “that sounds exhausting” without knowing anything about you, your situation, your city, your time zone, or whether the advice it is about to give is physically possible. That is the awkward part of emotional support AI: the tone can be soft while the facts are made of air. A very caring assistant can still recommend a midnight walk at 3 p.m., suggest a closed café, or confidently invent local details because it wants to be helpful. The kindness is real enough in style. The grounding is not. ...

A car assistant sounds simple until it touches the car. “Turn on the fan.” “Open the sunroof.” “Change my destination to Barcelona.” “Send an email before I arrive.” None of these requests looks philosophically difficult. They are not graduate-level math problems. They do not require poetic reasoning, legal interpretation, or a 128k-token context window stuffed with PDFs. They require the assistant to do something much less glamorous: check the state of the world, follow a few policies, use the right tools, and avoid pretending when something is missing. ...

Tower placement is a small decision until it is wrong. In a tower-defense game, a bad tower is not merely an inelegant plan. It is money spent, coverage lost, enemies leaked, and time wasted. The game does not care that the explanation sounded strategic. It only asks whether the tower actually touches the road. ...

A bad answer is easy to audit. It sits there, smug and wrong. A bad reasoning process is worse. It looks useful while it is drifting. It explains itself. It produces intermediate steps that sound locally plausible. It may even correct one mistake while preserving another, like a spreadsheet with a broken formula hiding behind tasteful formatting. ...

Training data has a bad habit: the easiest examples talk the loudest. Anyone who has trained a model on preference pairs knows the scene. One answer is clearly grounded in the image; the other confidently invents an object, a color, or an action that is not there. The model learns the contrast quickly. Everyone applauds. The loss goes down. The dashboard looks obedient. ...

TL;DR for operators FaithAct is useful because it changes the unit of control. Instead of asking whether a multimodal model’s final answer is correct, it asks whether each intermediate claim is supported by the image before that claim is allowed to steer the next step.1 That is a more operational target. Accuracy tells you whether the system arrived somewhere acceptable; perceptual faithfulness tells you whether it drove through the road or hallucinated a bridge. ...

TL;DR for operators FAITH is useful because it changes the hallucination question from “Does the model sound right?” to “Can the model reconstruct a known financial number from the exact tables and surrounding text that justify it?”1 That sounds modest. It is not. In finance, modest is usually where the damage hides. ...

TL;DR for operators A multimodal model can look at an image and still answer from memory, habit, or linguistic guesswork. That is the uncomfortable core of visual hallucination: the output is fluent, relevant-looking, and sometimes even useful, while being only loosely attached to the pixels it claims to describe. The practical lesson is not “never use multimodal AI.” That would be tidy, dramatic, and mostly useless. The lesson is narrower and more valuable: visual hallucinations need to be diagnosed by where grounding fails, not merely counted after the model has embarrassed itself. ...

TL;DR for operators LLM agents do not merely hallucinate by saying false things. They hallucinate when they act on a version of the world that does not match the task, the history, or the screen in front of them. That is the useful idea in MIRAGE-Bench: it treats agent hallucination as context-unfaithful action. The agent may click a button that is not there, assume a page transition succeeded when it did not, answer a colleague’s question with invented information, submit code despite failed tests, or report success when the environment says otherwise. Very industrious. Very confident. Very much not what you want near production systems. ...

TL;DR for operators When an LLM summarises a review, policy memo, support ticket, medical note, or news item, the operational question is not only “Did it get the facts right?” The sharper question is: did it change what the user is likely to believe, prioritise, or buy? The paper behind this article studies exactly that problem. It treats LLM-generated content as a decision interface and measures three ways the interface can quietly bend human judgment: changing the sentiment frame of the source, overemphasising the beginning of the source, and fabricating confident answers for events beyond the model’s knowledge cutoff.1 ...