Context-Compression

TL;DR for operators An agent can have a compressed memory that answers the current question perfectly and is already inadequate for the next update. Guangzhe Zhang’s Correct Now, Insufficient Later: Auditing Update Sufficiency in Context Compression1 demonstrates the problem with paired synthetic histories. On DeepSeek, the paper’s frontier selector and a latest-only variant both score 96/96 on current reads. After a later update makes older evidence relevant, frontier remains at 96/96, while latest-only falls to 32/96. ...

TL;DR for operators A retrieved prompt can contain one useful sentence or evidence distributed across several passages. Giving both requests the same compressed-context budget assumes that the amount of information worth preserving is constant. The evidence in ATACompressor1 suggests that assumption is costly: on MSMARCO with LLaMA-2-7B, removing adaptive allocation reduced F1 from 50.06 to 47.52 and compression ratio from 27.36x to 19.06x; removing selective compression reduced F1 further to 40.83. ...

TL;DR for operators Fofadiya and Tiwari propose a context-management system for long-running LLM interactions that does more than summarize old conversations.1 It scores historical turns by relevance, recency, dialogue dependency, and coherence, then applies a three-level policy: retain high-value turns verbatim, summarize medium-value turns, and remove low-value history when the context budget requires it. ...

A codebase is not a paragraph. Neither is a litigation folder, a clinical case file, a customer-support history, a policy archive, or the slow-motion disaster known as “all meeting notes since March.” Yet many enterprise AI systems still treat long context as a heroic prompt-engineering problem: push more text into the model, pray the key detail survives attention, and call the bill “innovation.” ...

Opening — Why this matters now The AI industry has developed a charmingly expensive habit: when models struggle with long documents, we buy them larger windows and pretend the problem has been solved. It has not. Long-context LLMs are useful, but longer context is not the same as better context. A model can accept a very large input and still miss the crucial paragraph buried in the middle, over-attend to duplicated evidence, or lose the argumentative spine of a document. The result is familiar to anyone building AI tools for legal review, finance research, policy analysis, procurement, consulting, compliance, or enterprise knowledge work: the model has “read” everything, yet somehow understands the wrong thing. Very modern. Very expensive. ...

Long context is a lovely product promise until the invoice arrives. Every enterprise AI demo eventually wants the same magic trick: read the whole contract archive, remember every customer interaction, inspect every ticket, keep all meeting notes alive, and answer as if the model has a tidy brain instead of a very expensive attention matrix. The sales slide says “128K context.” The infrastructure team hears “latency, memory, and GPU burn.” Both are correct. One is merely dressed better. ...