Benchmarks are clean. Research is not. A benchmark asks a model to answer a question, then politely stops. A research workflow asks the model to form a hypothesis, test it, read the result, notice what went wrong, adjust the plan, and try again without wandering into scientific nonsense. One is a quiz. The other is a beaker with a budget, a deadline, and a surprisingly expensive simulation queue. ...
The invoice arrives after the benchmark party Math benchmarks are fun until the training bill arrives. A model can be taught to produce longer reasoning traces. It can be shown more olympiad problems. It can be given Python. It can be pushed into 128K-token contexts and told, heroically, to think harder. All of this sounds impressive in a benchmark table. Less impressive is the operational detail that most training samples do not need the full 128K window, yet a naive training setup can still make every step pay for it. ...
Audit is usually boring until the system being audited can write a beautiful excuse. Ask a language model why it refused a harmful request, why it used a shortcut, or why it made a strange numerical mistake, and it may give a polished answer. That answer may even sound morally mature, procedurally clean, and delightfully compliant with the safety policy. Very nice. Also: not enough. ...
Search is not judgment Search is easy to admire because it produces something visible. A ranked list. A bigger context window. A satisfying pile of passages that says, “Look, we retrieved evidence.” Very comforting. Also not the same as knowing what evidence is actually needed. That distinction is the core of Context-Picker: Dynamic Context Selection Using Multi-stage Reinforcement Learning.1 The paper studies a familiar RAG problem: if a system retrieves too little, it misses the answer; if it retrieves too much, it drags in distractors, repeats, weakly related fragments, and the usual long-context swamp where useful evidence politely disappears in the middle. ...
Losing is not the problem. Being seen losing is. Put two AI agents in the same workflow and the design immediately stops being a simple productivity question. One agent writes code. Another reviews it. A third ranks alternatives. A fourth routes the next task to whoever looks most competent. At the slide-deck level, this is “multi-agent collaboration.” In the logs, it is often a scoreboard with better manners. ...
Diagnosis is not a magic word. In medicine, the answer matters, but the path to the answer matters almost as much. A model that says the correct disease name after skipping the decisive evidence is not “reasoning efficiently.” It is guessing with bedside manner. That is the problem addressed by MedCEG: Reinforcing Verifiable Medical Reasoning with Critical Evidence Graph.1 The paper’s core claim is not simply that a medical LLM can score higher on benchmarks. That would be useful, but not especially surprising. The more interesting move is architectural: the authors try to make clinical reasoning trainable by turning it into a graph of required evidence, then rewarding the model for following that graph. ...
A patient walks into a clinic and tells the doctor several things at once: chest tightness, shortness of breath, leg swelling, leg pain, maybe a history of walking too much, maybe some anxiety, maybe something that sounds more obviously cardiac. The dangerous part is not the word “chest.” The dangerous part is the chain: leg swelling and pain may suggest deep vein thrombosis; shortness of breath may suggest pulmonary embolism; pulmonary embolism can kill. ...
A farmer does not need a literature review. She needs to know what works. That simple sentence is why AI agronomy is so tempting. Somewhere inside thousands of papers are useful answers: which microbial agents suppress whitefly, whether botanicals work outside the lab, how much pest control disappears when a method leaves a greenhouse and meets weather, soil, and actual insects with their own little business plans. The evidence exists, but it is fragmented, multilingual, paywalled, and written in the soothing dialect of “further research is warranted.” ...
The inbox problem hiding inside RAG Inbox. That is the easiest way to understand what goes wrong in many retrieval-augmented generation systems. A query arrives. The system retrieves a few documents. The answer is not obvious. So the system retrieves more. Then more. Then perhaps a web search result. Then a rewritten query. Then another bundle of passages. ...
Shopping looks easy until someone has to calculate the customs duty. That is roughly the lesson of EcomBench, a new benchmark designed to evaluate foundation agents on realistic e-commerce tasks.1 The paper’s most useful finding is not that one model ranks above another. Leaderboards are entertaining, in the same way airport departure boards are entertaining when your flight is already delayed. The useful finding is the shape of failure. ...