Cover image

Both Alleles Scored 0.995. The Experiment Disagreed.

TL;DR for operators A scientific AI system should not treat a computational prediction as a verified finding simply because an LLM can explain it or retrieve supporting annotations. ARGUS demonstrates an alternative architecture: prediction models generate hypotheses, an agent retrieves relevant observations, deterministic rules determine scientific status, and an LLM communicates conclusions within explicit evidence constraints. ...

October 30, 2026 · 8 min · Zelina
Cover image

A Shorter Prompt Is Only as Safe as Its Test

TL;DR for operators Production-agent prompts can be shortened without a measurable decline in benchmark correctness, but the benchmark must first be trustworthy. A documented optimization failure removed important instructions while passing sampled checks, exposing a risk that aggregate scores can conceal. A new study addresses this through reproducible evaluation of agents connected to changing business data, followed by correctness-gated prompt optimization. Across three model families, it reports 3.5–7.6% reductions in the complete instruction surface, with a statistically significant correctness improvement for one family. ...

October 29, 2026 · 8 min · Zelina
Cover image

The Right Answer Was in the Pool

TL;DR for operators An AI system can generate the correct answer and still return an incorrect one. The distinction matters when teams spend additional inference resources producing multiple candidate solutions but rely on an imperfect mechanism to choose among them. Xiao Yue and Guangzhi Qu introduce Coverage, Realization, and Validity Evidence (CRV), a diagnostic protocol for separating correct-answer availability, answer-selection performance, and evidence that a correct-answer derivation is supported by its premises.1 ...

October 29, 2026 · 8 min · Zelina
Cover image

A World That Ends Correctly Can Still Be Wrong: Testing LLM Agents After Delivery

TL;DR for operators In World Agent: Can Language Models Keep a World Running?1, the central problem is not whether an agent can produce the expected event. It is whether those events remain correctly organized as the world evolves. Across the benchmark, models often complete more required events than they correctly satisfy the relationships among those events. Causal-relation checking is especially fragile: its cross-model mean falls from 0.683 in the Easy maintenance tier to 0.356 in Extreme, where substantially more of the world representation must be constructed by the agent. ...

October 6, 2026 · 7 min · Zelina
Cover image

Following Instructions Is Not the Same as Knowing More

TL;DR for operators A multimodal model can become much better at obeying instructions without becoming much better at the underlying tasks those instructions govern. In the experiments examined here, one 8B vision-language model gains 10.58 percentage points on a targeted instruction-following benchmark and another gains 22.91 points. Yet their average results across broader STEM, VQA, OCR, and document-understanding tests move by only +0.33 and -0.21 points. ...

October 1, 2026 · 7 min · Zelina
Cover image

Before the Test Comes the Question: The LLM Formulation Gap in Analytics

TL;DR for operators An analytics copilot can know statistics and still start from the wrong problem. StatFormBench tests what happens before statistical execution: given an informal request and heterogeneous data, can an LLM determine the statistical problem being asked, select the data objects that actually matter, and assign each one the correct analytical role? Across 1,013 human-reviewed scenarios and 14 LLMs, those abilities do not move together. Gemini 3.1 Pro achieves the highest fine-grained problem-classification accuracy at 72.0, while Claude Opus 4.6 achieves the highest variable-set overlap at 63.2. No evaluated model leads both components. ...

September 30, 2026 · 8 min · Zelina
Cover image

The Graph Isn’t the Verifier: What LCoT-GV Actually Learns From Long Reasoning Chains

TL;DR for operators A long reasoning trace creates an additional quality-control problem: even when the reasoning looks structured, the final answer can still be wrong. A verifier therefore has to identify signals inside the trace that predict answer correctness without simply trusting the model that produced it. LCoT-GV, introduced by Bérénice Jaulmes and Mehwish Alam,1 turns reasoning steps into a graph, connects steps when a local inference model judges them to support or contradict one another, and then uses a graph attention network to classify whether the final answer is correct. Across three reasoning models, its default configurations average 75.24%-77.92% accuracy. ...

September 29, 2026 · 7 min · Zelina
Cover image

Zero Is Not Nothing: When an Inactive Token Still Changes the Model

TL;DR for operators If adding an unused token whose hidden state is exactly zero changes existing tokens or predictions, that token is not actually inactive. Standard softmax attention can produce exactly this behavior: a visible zero source still contributes normalization mass even when its value contribution is zero. OAttention1 addresses this at the attention operator by giving each token a norm-derived presence coefficient. The coefficient gates what a receiver emits and, separately, how much support a source contributes to the attention numerator and denominator. Exact zero then means zero participation under the declared operator contract. ...

September 24, 2026 · 6 min · Zelina
Cover image

The Graph Is Not the Guardrail: Route Retrieval by Failure Mode

TL;DR for operators A retrieval system may have several ways to answer the same analyst request: search semantically similar text, follow explicit relationships in a knowledge graph, repair a failed graph query, or combine graph and text evidence. The operational question is not which technique has the highest average score. It is which path fails acceptably for the workload in front of it. ...

September 19, 2026 · 8 min · Zelina
Cover image

More Thought Is Not Always More Reliable: Routing Reasoning for Social Judgment

TL;DR for operators Extra inference compute is not a monotonic reliability upgrade for socially ambiguous tasks. Across three benchmarks, reasoning-focused models sometimes outperform non-reasoning counterparts, sometimes underperform them, and can become less accurate when deliberation is pushed harder on difficult cases. The practical lesson is not to suppress reasoning. Moderate reasoning, token limits, and adaptive stopping can improve results. Instead, treat reasoning depth as a control variable: decide when to invoke it, when to stop it, and whether the prompt format itself is steering the model toward shortcuts. ...

September 17, 2026 · 7 min · Zelina