Cover image

When the Newest AI Skill Isn't the Best

TL;DR for operators An AI agent can fix one visual-reasoning failure and introduce another. As its instructions and tools evolve, operators need a way to distinguish genuine capability improvements from revisions that merely solve recent examples. They also need to diagnose recurring failures without repeatedly examining entire execution histories. VisualEvolve addresses both problems through versioned skill libraries and structured evidence retrieval. Across four frozen vision-language models and six evaluation sets, it improves over the initial skill library in 23 of 24 comparisons, with one tie. Removing historical version selection or structured evidence access substantially reduces performance. For organizations maintaining AI agents, the research supports treating skill revisions as evaluated, reversible artifacts rather than automatically accepting the latest changes. It does not yet establish enterprise reliability or lower total operating costs. ...

October 31, 2026 · 8 min · Zelina
Cover image

A Good Score Is Not Permission to Rewrite the Playbook

TL;DR for operators An agent that repeatedly uses a procedure and receives high task reward still has not shown that the procedure deserves permanent promotion into its reusable playbook. R² Flow separates three questions that agent systems often blur together: how much a skill participates in successful execution, whether choosing that skill actually improves outcomes relative to alternatives, and whether there is enough independent evidence to authorize a persistent library change. Across eight recursive phases, the paper reports mean held-out OOD score rising from 71.2 to 81.02, with 33 of 37 committed edits improving held-out verified score. By contrast, reward-derived edit labels can raise training reward while lowering that verified score. ...

October 2, 2026 · 9 min · Zelina
Cover image

Skill Issue, Literally: Repairing Agent Instructions Without an Answer Key

TL;DR for operators Runbooks decay. APIs shift, data schemas mutate, file paths move, and the “expert procedure” that worked last quarter starts quietly steering an agent into a wall. The paper behind this article, SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing, asks a useful operational question: can an agent skill be improved when nobody has provided hidden tests, reference answers, task rewards, or expert labels?1 ...

July 3, 2026 · 21 min · Zelina