Cover image

Blocked Is Not Stopped

TL;DR for operators A runtime monitor can correctly block a prohibited tool call and still fail to enforce the intended policy. In EvasionBench, agents receive legitimate tasks, encounter an explicit guardrail denial, and are allowed to continue working. Across ten evaluated agents, the highest best-of-three evasion success rate is 88%, while possible evasion-attempt rates reach 98%. ...

October 16, 2026 · 8 min · Zelina
Cover image

Spend the Next Test Where the Evidence Points

TL;DR for operators A fixed red-team suite answers a predetermined set of questions. CART asks an additional operational question: given what the last tests revealed, where should the next test budget go? In an illustrative 99-round run, the first 12 reconnaissance tests produced zero scores. A cascading-hallucination signal appeared at round 13. From there, 67% of the remaining probes were directed toward the two most brittle categories, while testing strategies continued to vary; the peak risk score reached 4.37. ...

October 13, 2026 · 8 min · Zelina
Cover image

The Right Number Can Still Answer the Wrong Question

TL;DR for operators A financial agent can retrieve the correct filing, extract a defensible figure, ask a relevant follow-up question—and still return the wrong answer for what the user meant. The clearest evidence is the gap between knowing the interpretation and having to recover it. When evaluated models are supplied with the disambiguating interpretation, their accuracy rises to roughly 90-95%. Under full free interaction, where the agent must decide what to clarify and use the answer correctly, the best result is 28.9%. ...

October 13, 2026 · 7 min · Zelina
Cover image

When One Bad Memory Hides Another

TL;DR for operators A persistent-memory agent can retrieve two harmful records that substitute for one another. Remove either record alone, and the remaining one can preserve the same wrong answer, making both look harmless under a one-at-a-time deletion test. MeClear1 addresses that failure mode by measuring each retrieved memory across multiple combinations of the surrounding context rather than only against the full retrieved set. It uses sampled Shapley contributions to identify negatively contributing memories, ranks them, and then tests nested clearance candidates. A nonempty clearance is accepted only when it produces positive empirical task gain and passes a fixed recovery check. ...

October 12, 2026 · 7 min · Zelina
Cover image

Same Facts, Different Price

TL;DR for operators A pricing agent can receive the same underlying market facts and still behave differently because those facts arrive in a different format, order, or textual context. Lee and Park call this market signal injection: presentation-level manipulation of LLM pricing inputs without an explicit instruction to set a particular price.1 Their simulations show that the effect can extend beyond the manipulated agent. Under undefended sentiment-context attacks, non-target firms’ mean price shifts were about 83–99% of the target firm’s shift in duopoly and 65–99% in triopoly. ...

October 10, 2026 · 8 min · Zelina
Cover image

The Model Saw the Paper. The Attribution Still Failed.

TL;DR for operators In RINI: Seeing the Prior Is Not Enough1, a research proposal can be shown the paper that already establishes its claimed technical contribution, recognize that paper as relevant, and still claim the contribution as its own. Among 175 interpretable proposals exposed to such evidence, 137 recognized the prior-target relationship, but only 61 correctly assigned the established contribution to the prior. ...

October 9, 2026 · 7 min · Zelina
Cover image

A Smaller Agent Is Not the Same Agent: Training for Deployment Perturbations

TL;DR for operators Wang et al. study a deployment problem that is easy to underestimate: an RL-trained agent can work well before compression or modification and then fail badly after its policy is perturbed.1 Their proposed method, Stable Perturbation-Robust Policy Optimization (SPrPO), trains the agent under perturbations while allocating less noise to locally sensitive FFN channels and adjusting total perturbation strength using a gradient-based improvement proxy. ...

October 7, 2026 · 7 min · Zelina
Cover image

A World That Ends Correctly Can Still Be Wrong: Testing LLM Agents After Delivery

TL;DR for operators In World Agent: Can Language Models Keep a World Running?1, the central problem is not whether an agent can produce the expected event. It is whether those events remain correctly organized as the world evolves. Across the benchmark, models often complete more required events than they correctly satisfy the relationships among those events. Causal-relation checking is especially fragile: its cross-model mean falls from 0.683 in the Easy maintenance tier to 0.356 in Extreme, where substantially more of the world representation must be constructed by the agent. ...

October 6, 2026 · 7 min · Zelina
Cover image

Correct Now Is Not Enough: Testing Agent Memory for What Happens Next

TL;DR for operators An agent can have a compressed memory that answers the current question perfectly and is already inadequate for the next update. Guangzhe Zhang’s Correct Now, Insufficient Later: Auditing Update Sufficiency in Context Compression1 demonstrates the problem with paired synthetic histories. On DeepSeek, the paper’s frontier selector and a latest-only variant both score 96/96 on current reads. After a later update makes older evidence relevant, frontier remains at 96/96, while latest-only falls to 32/96. ...

October 6, 2026 · 6 min · Zelina
Cover image

Make Verification Expensive: RedHerring Turns an Agent’s Search Budget Into a Security Surface

TL;DR for operators An autonomous security agent can notice far more suspicious code paths than it can fully prove vulnerable during a fixed audit window. In the paper’s 300-trajectory analysis, verification consumed 73.8% of wall-clock time, compared with 7.3% for hypothesis formation; verifying one hypothesis took about 11.1 times as much time as generating one. ...

October 5, 2026 · 7 min · Zelina