Cover image

The Verdict Is Not the Evidence

TL;DR for operators In Beyond QASecClaw: Agent Scaling, Verifier Feedback, and Evidence Reliability in Software Assurance1, additional LLM assessment contexts did not consistently make defect-clearance decisions safer. Scaling from one to 50 contexts increased token consumption approximately 49-fold on eight authored Java warning controls without meaningfully reducing unsafe clearance. Raw formal-verifier feedback improved correct assertion verdicts but also increased incorrect ones. More surprisingly, models frequently rejected valid counterexamples that correctly demonstrated program defects. ...

November 11, 2026 · 10 min · Zelina
Cover image

Train the Advice, Not Just the Agent

TL;DR for operators A multi-step AI agent may retrieve useful information, lose track of a task constraint, or continue searching after it already has enough evidence. A separate advisor can recommend the next action, but asking a larger model for feedback does not guarantee that the advice will improve task completion. Caddie1 trains an advisor using what happens after the acting agent follows its suggestions. On MuSiQue with supplied passages, a Qwen3-4B agent improves from 21.56% success without assistance to 46.88% with Caddie. A prompted Kimi K3 critic reaches only 29.13% in the same comparison. The acting model remains unchanged; reinforcement learning updates the advisor. ...

November 7, 2026 · 9 min · Zelina
Cover image

One Harness, Many Failure Modes

TL;DR for operators A platform team supporting several language models can achieve respectable task coverage while still delivering inconsistent results across those models. The relevant question is not only whether some supported model completes a task, but whether the same agent harness works reliably with every supported model. MaD-RSI, proposed by Yu Cui and colleagues,1 addresses this problem by using execution successes and failures from heterogeneous, frozen LLMs to improve one shared harness. On MedXpertQA, it matches the strongest single-model optimization baseline’s 60% four-model success coverage while raising joint success from 13% to 18%. ...

November 3, 2026 · 8 min · Zelina
Cover image

Same Actions, Different Outcomes: The Cost of Tool-Schema Bias

TL;DR for operators Changing how an agent calls tools can change its task success even when the underlying operations remain identical. In a controlled study, one model’s success fell from 78.5% to zero after tools were consolidated into a particular interface representation, then recovered to 75.2% under another merged representation. Across nine main open-weight models, fully merging tools reduced success by a median of 19 percentage points. ...

October 23, 2026 · 9 min · Zelina
Cover image

The Agent's Second Record: What Its Tools Actually Execute

TL;DR for operators An enterprise agent can invoke a legitimate-looking tool while its underlying process accesses credentials, launches additional commands, or transmits data. The conversation transcript may never reveal those side effects. But monitoring execution alone can also miss attacks that manipulate what the agent reads or decides. The Agent Cross-Layer Evidence (ACE) benchmark shows why the distinction matters. For silent exfiltration through logging, application evidence barely distinguishes malicious sessions (AUROC 0.586), while operating-system evidence performs strongly (0.976). For retrieval-reference injection, the relationship reverses: application evidence reaches 0.906 versus 0.703 for kernel evidence. These are mean in-distribution results across 10 learned detectors. Combining both evidence sources leads all 10 detectors in-distribution and seven under held-out-attack-family evaluation, although kernel-only evidence wins for some detectors and attack mechanics. ...

October 23, 2026 · 8 min · Zelina
Cover image

When Old Instructions Outlive Their Authority

TL;DR for operators An AI agent can retain every instruction in a conversation and still execute the wrong task. The problem becomes acute when users replace requirements, withdraw constraints, or revise decisions on which earlier work depended. A controlled study introduces IntentFlux, a benchmark that measures this failure under changing user instructions.1 In a turn-matched experiment, mean task score falls from 0.734 without revisions to 0.469 with revisions. Additional conversation length alone cannot explain the difference. ...

October 22, 2026 · 8 min · Zelina
Cover image

Reasoning Doesn't Cancel the Nudge

TL;DR for operators An autonomous purchasing agent may be choosing between two reasonable products while the interface quietly changes the decision environment: one item is preselected, or one is presented as socially popular. A randomized experiment with 3,600 GUI agents and 21,600 product choices found that both interventions strongly shifted what agents selected. More strikingly, increasing the agents’ reasoning setting pushed the two effects in opposite directions. Within the default condition, high reasoning roughly halved the odds of selecting the nudged product. Within the social-influence condition, high reasoning increased those odds by about 71%.1 ...

October 19, 2026 · 7 min · Zelina
Cover image

Write the Memory Before You Search It

TL;DR for operators A long-running assistant can store the right fact and still fail to retrieve it because that fact was represented alongside several unrelated meanings. AutoViewMem1 addresses this before search begins: it learns complementary ways to interpret a user’s history, writes structured memories under those views, removes redundancy, and then uses ordinary top-K vector retrieval. ...

October 19, 2026 · 8 min · Zelina
Cover image

A Like Can Rewrite the Feed

TL;DR for operators An autonomous agent that likes recommended content can change its own future information environment. The Like Trap: Multi-Stage Poisoning against Agents in Similarity-based Recommendation Systems1 shows how an attacker can exploit that loop in a similarity-based recommender: early poisoned posts are close enough to the user profile to be surfaced and liked, then those likes increase the scores of later poisoned posts that are progressively less similar to the user. ...

October 18, 2026 · 7 min · Zelina
Cover image

Before You Retry the Agent, Check the Evidence

TL;DR for operators An agent can already have the information needed to answer a question and still produce the wrong answer. Diehl and Lim’s Evidence-Inference Reconstruction (EIR)1 addresses that failure mode by storing retrieved source sentences separately from the agent’s evolving interpretations, then generating the scored answer in a fresh call that sees only the original question and accumulated evidence. ...

October 18, 2026 · 7 min · Zelina