When Vision Fails in Both Directions
TL;DR for operators A model can correctly recognize that one object is left of another and still fail when asked to generate that same relationship. More importantly, some visual weaknesses recur in both directions. AMVICC maps visual-language understanding and image generation onto visual concepts derived from the same underlying benchmark. Across the tested systems, weaknesses repeatedly appear in Quantity and Count, Positional and Relational Context, Orientation and Direction, and State and Condition. Text behaves differently: most tested multimodal language models avoid the paper’s Text failure threshold, while all three tested image generators fall below it for explicit generation. ...