Multimodal AI

Cameras are easy. Audits are not. That is the useful irritation inside FoodMonitor: Benchmarking MLLMs for Explainable Compliance Analysis, a new benchmark for testing multimodal large language models on commercial-kitchen compliance monitoring.1 The paper is not asking whether a model can watch a kitchen video and say something vaguely sensible about hygiene. Many systems can now do that, at least with enough confidence to impress a demo audience and mildly alarm the legal department. ...

Video is easy to collect and expensive to understand. That is the awkward little truth behind many enterprise “AI video intelligence” projects. A warehouse camera records everything. A body camera records everything. A meeting room system records everything. A field-service headset records everything. Then someone asks a very human question: who handled the device after lunch, what did they say, and was the machine hot when they touched it? ...

Enterprise AI has developed a charming habit: whenever a system fails, someone suggests using a larger model. The chatbot misread a customer complaint? Bigger model. The autonomous system struggled with a new sensor configuration? Bigger model. The video classifier understood the objects but missed the actual message? Bigger model, possibly with a more expensive logo. ...

A dashboard is rarely confusing because the pixels are ugly. More often, the problem is that the important part is small, crowded, rotated, hidden in a chart corner, split across spatial relations, or buried inside a scene that needs to be mentally transformed before the answer becomes obvious. A human analyst zooms, marks, traces, rearranges, or imagines a new angle. A multimodal model, by contrast, is often asked to stare at the original image and talk harder. ...

Immersive AI has a convenient myth: put a stronger multimodal model inside a headset, let it see what the user sees, and the future of work politely appears. Very cinematic. Slightly incomplete. The real problem is less glamorous and more operational. Extended-reality work is not just a visual scene. It is a long-running loop of perception, memory, reasoning, instruction, correction, confirmation, and physical effort. The model must understand what is happening over time. The human must still steer the system without becoming a tired thumb attached to a battery pack. ...

Reasoning usually arrives as text. A model explains itself in sentences, equations, bullet points, and the occasional theatrical “therefore.” We have learned to call this chain-of-thought, or CoT, because “the model wrote a long scratchpad and we hope it helped” sounded insufficiently scientific. The paper Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text asks a sharper question: what if the intermediate reasoning medium does not have to be text at all?1 ...

Multimodal AI has reached the point where it can describe videos, summarize documents with images, answer visual questions, and generate outputs that look satisfyingly complete. This is exactly why evaluation is becoming more dangerous. A system that looks competent is not necessarily reliable. It may miss the one-second event that determines the answer. Or it may notice enough evidence but then produce a fluent, attractive, visually decorated summary that quietly distorts the facts. The first failure is upstream: the model did not capture the decisive evidence. The second is downstream: the output did not preserve and present the evidence in a human-useful way. ...

A document lands in an intake queue. It might be an invoice, a memo, a form, a résumé, or one of those corporate artifacts whose layout says more than the words do. Someone wants the system to classify it instantly, because every downstream workflow—routing, extraction, compliance, archiving—depends on that first label. The fashionable answer is: send it to a large language model. Extract the text, paste it into a prompt, ask for one label, and let the machine be clever. This is attractive because it feels general. It is also how many automation projects quietly turn a visual problem into a text problem, then act surprised when the system starts calling file folders “proposals” because the word proposal appeared somewhere on the page. ...

Pixels to Purchase Orders: A Business Map for Choosing Vision-Language Models Receipts are a good way to ruin an AI demo. A clean product photo is polite. A scanned receipt is not. It has shadows, folds, strange fonts, tiny numbers, merchant abbreviations, table-like structure, and one suspiciously important total amount hiding near the bottom. Ask a generic multimodal assistant what it sees, and it may produce an answer that sounds fluent enough to make everyone in the meeting relax. That is usually the dangerous part. ...

A slide looks finished. The headline is sharp, the equations are aligned, the answer box is confident, and the design has the mild corporate glow of something that has already been approved by three people who did not read it. That is exactly the problem. For years, text-to-image models failed in a wonderfully obvious way: they could not spell. A poster would say “Qaurterly Reveneu,” the mockup button would contain mystical glyphs, and everyone understood the output was decorative, not operational. Recent models have changed that. They can now place readable text inside images, produce document-like pages, and generate slide-like visual artifacts. The failure mode has become less funny and more expensive: the text may be readable, but the reasoning may be wrong. ...