Visual Grounding

TL;DR for operators Longer reasoning is not automatically safer for vision-language systems. This paper finds that a model can preserve the relevant visual information yet perform worse once reasoning shifts attention away from the image region that matters. The paper separates two questions that are often conflated in multimodal evaluation: how much attention goes to the image, and whether that attention is concentrated on evidence relevant to the question. Targeted masking experiments suggest that the heads with concentrated visual focus are functionally important, not merely correlated with success. ...

Sight Unseen: How LVLM Alignment Can Teach Models to Ignore Images Image inspection has one rude requirement: the model should look at the image. That sounds too obvious to be an article thesis, which is usually a warning sign. In real deployments, a large vision-language model may describe a damaged package, summarize a product photo, inspect a dashboard screenshot, answer a question about an invoice, or guide a visual agent through a web interface. When it gets something wrong, the default diagnosis is familiar: the vision encoder missed the object, the dataset was noisy, the benchmark was weak, or the model simply hallucinated because models hallucinate. Very tidy. Also incomplete. ...

A model can see the image and still miss the point Inspection is a wonderfully cruel test for AI. Show a multimodal model a product photo, a medical scan, a factory defect, a form, or a dashboard screenshot, and the answer may sound calm, fluent, and technically plausible. The model may even imitate the reasoning style of a stronger teacher model. It may describe objects, infer relationships, and produce the correct-looking sentence. ...

Screenshots look simple until they are not. A human opening a dense professional application does not inspect every pixel with equal seriousness. We glance, zoom in mentally, ignore decorative clutter, search for the likely region, then focus. In other words, we do not merely “see” the interface. We decide where to look. ...