Grounding

Correct Answer, Weak Evidence: Measuring Multimodal Reasoning at the Fact Level

TL;DR for operators A correct multimodal answer with citations can still have a weak evidence trail. In human evaluation on Video-MMMU, Gemini-3-Pro reached 86% question-answering accuracy but only 16.3 on MuRGAt-Score, meaning answer performance and fact-level grounding ranked very differently. MuRGAt evaluates two requirements separately. Coverage asks whether verifiable claims receive citations. Attribution asks whether the cited frames, audio, figures, or temporal segments are actually relevant to and sufficient for the factual claims they accompany. The combined score rewards systems only when both conditions hold. ...

Higher Pass Rate, More Broken Tasks: The Regression Tax in Agent Skill Libraries

TL;DR for operators Adding reusable instructions to an agent creates a release-management problem that average accuracy does not fully expose. A library can solve tasks the baseline missed while simultaneously breaking tasks the baseline handled correctly. Darshan Tank and Baran Nama measure that trade-off directly in The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents.1 Across 18 skill-library conditions, they observe 553 baseline-fail-to-skill-pass transitions but also 324 baseline-pass-to-skill-fail transitions. The new failures offset 59% of the gross gains. ...

Don’t Tell the Robot What You Know

Directions are easy when both people see the same room. “Move left.” “Go toward the table.” “The apple is beside the sofa.” These are perfectly reasonable instructions if speaker and listener share the same visual world. They become less reasonable when one of them is staring at a wall, cannot see the table, and has no reason to believe the sofa exists. At that point, the problem is no longer navigation. It is epistemology, with furniture. ...

Grounded or Just Confident? What the AI Consumer Index Reveals About Frontier Models

Shopping is where AI confidence goes to embarrass itself. Ask a frontier model for a gift, a replacement part, a budget-friendly product, or a game recommendation, and the answer often looks excellent. It is neatly formatted. It gives reasons. It may even include links and prices, because apparently nothing says “trust me” like a fabricated discount on a product page that no longer exists. ...