Document AI

TL;DR for operators Most AI performance failures are not solved by scaling the most visible knob. Three recent papers make the same uncomfortable point from different angles. A controlled image-classification study finds that more data gives more stable generalization gains than simply increasing model complexity, while added visual priors help only when the architecture can use them.1 A document parsing benchmark shows that frontier VLMs and specialized parsers still fail on expert documents with dense layouts, formulas, tables, music notation, rotation, and long-document reading order.2 A LoRA optimization paper argues that adapter performance is often limited not by rank alone, but by a mis-scaled LoRA scaling factor, usually treated as a small implementation detail because apparently we needed another reminder that details run the building.3 ...

A document lands in an intake queue. It might be an invoice, a memo, a form, a résumé, or one of those corporate artifacts whose layout says more than the words do. Someone wants the system to classify it instantly, because every downstream workflow—routing, extraction, compliance, archiving—depends on that first label. The fashionable answer is: send it to a large language model. Extract the text, paste it into a prompt, ask for one label, and let the machine be clever. This is attractive because it feels general. It is also how many automation projects quietly turn a visual problem into a text problem, then act surprised when the system starts calling file folders “proposals” because the word proposal appeared somewhere on the page. ...

Pixels to Purchase Orders: A Business Map for Choosing Vision-Language Models Receipts are a good way to ruin an AI demo. A clean product photo is polite. A scanned receipt is not. It has shadows, folds, strange fonts, tiny numbers, merchant abbreviations, table-like structure, and one suspiciously important total amount hiding near the bottom. Ask a generic multimodal assistant what it sees, and it may produce an answer that sounds fluent enough to make everyone in the meeting relax. That is usually the dangerous part. ...

Screenshots lie differently from HTML. That sounds like a small engineering nuisance until the model is not merely answering a demo question, but reading a supplier invoice, comparing products on a procurement portal, interpreting a dashboard, or deciding which button an autonomous web agent should click next. The same underlying object may appear as a rendered page, raw DOM, OCR text, chart pixels, table JSON, or a caption. Humans usually treat these as different windows onto the same thing. Multimodal models often treat them as different worlds. ...

A dashboard screenshot is often too little. A video walkthrough is often too much. Somewhere between the two sits a strangely old-fashioned interface: panels, captions, arrows, speech bubbles, and a sequence that tells the machine what happened before what. Yes, comics. That sounds unserious only if we think comics are a decoration layer: something added after the reasoning is complete to make the output friendlier. The paper Thinking with Comics: Enhancing Multimodal Reasoning through Structured Visual Storytelling makes a more interesting claim: comics can act as the reasoning medium itself, not merely the illustration of reasoning after the fact.1 ...

A scanned exam paper is not a polite input. It arrives bent, shadowed, annotated, folded, half-covered by a student’s handwriting, and occasionally photographed at an angle chosen by someone apparently in active conflict with geometry. For a human teacher, this is annoying. For a document AI system, it is more than annoying. It creates a dangerous fork in the road: extract what is visible, or admit that the question cannot be recovered. ...

A receipt is not hard to understand because it is philosophical. It is hard because the answer may live in one corner, the label in another, and the meaning in the relationship between them. That is exactly the kind of thing multimodal large language models are supposed to be getting better at. Give the model an image. Ask a question. Let the model inspect the pixels and reason over the scene. The product demo looks magical until the model reads the wrong number, misses the column header, confuses the parking space for a lane, or confidently answers a chart question from the wrong local patch. Then the magic becomes a support ticket. ...

Reports are meant to be read. ESG reports, unfortunately, are often meant to be admired, navigated, skimmed, quoted, selectively screenshotted, and occasionally endured. They arrive as glossy PDFs full of charts, tables, diagrams, narrative claims, compliance language, decorative layout choices, and headings that may or may not behave like headings. The result is a familiar corporate ritual: a firm publishes hundreds of pages of sustainability disclosure, investors and regulators ask what it means, and everyone quietly discovers that the document is more presentation object than data infrastructure. ...

TL;DR for operators Long-document translation does not fail only because the model lacks enough tokens. It fails because documents are not bags of sentences. They contain references, implied pronouns, repeated terms, topic shifts, callbacks, causal links, and the occasional sentence that makes sense only because something three paragraphs earlier did the heavy lifting. ...

TL;DR for operators MemAgent is not another “look, we made the context window enormous” paper. Thank goodness; the context-window arms race was starting to look like cloud billing cosplay. The paper’s core move is simpler and more interesting: take a standard dense transformer, let it read a long document in chunks, and force it to maintain a fixed 1024-token working memory. After each chunk, the model overwrites that memory. At the end, it answers using the problem and the memory, not the whole document. The authors then train this behaviour with reinforcement learning, so the model learns what to retain, what to discard, and when a piece of information is merely shiny garbage. ...