Agent Skills

TL;DR for operators Adding reusable instructions to an agent creates a release-management problem that average accuracy does not fully expose. A library can solve tasks the baseline missed while simultaneously breaking tasks the baseline handled correctly. Darshan Tank and Baran Nama measure that trade-off directly in The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents.1 Across 18 skill-library conditions, they observe 553 baseline-fail-to-skill-pass transitions but also 324 baseline-pass-to-skill-fail transitions. The new failures offset 59% of the gross gains. ...

TL;DR for operators Runbooks decay. APIs shift, data schemas mutate, file paths move, and the “expert procedure” that worked last quarter starts quietly steering an agent into a wall. The paper behind this article, SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing, asks a useful operational question: can an agent skill be improved when nobody has provided hidden tests, reference answers, task rewards, or expert labels?1 ...

A workflow breaks in a boring way. The agent found the website yesterday. Today the button moved. Yesterday it parsed the file path correctly. Today the file name has a space, a date, and some human creativity sprinkled in for punishment. Yesterday the chart script worked. Today the data source changed its column names because apparently stability was not on the roadmap. ...

Compliance has a wonderful way of killing beautiful demos. In a demo, the agent calls a frontier model, loads a tool, reads a document, writes a decision, and everyone nods at the future. In a regulated company, the same workflow meets a less poetic checklist: where did the data go, who pays for the GPU time, can this run inside our perimeter, and why did the model spend twenty seconds “thinking” about a binary classification task? ...