The Skill Was Retrieved, Not Proven

TL;DR for operators

An agent completes a task after retrieving several reusable instructions. Its execution log records the instructions and the successful outcome, but not which instruction actually helped. Keeping every instruction associated with success can preserve redundant or harmful guidance; removing instructions with weak performance records can eliminate capabilities the model still needs.

TrackRL, introduced in Trackable Skill Library Lifecycle Management for Agentic Reinforcement Learning,¹ addresses this attribution problem by randomly including or withholding individual skills across attempts at the same task. It then separately measures whether the agent succeeds without the skill. These measurements support different decisions about retention, retirement, and harmful removal.

The distinction matters experimentally. On seen ALFWorld tasks, removing individual skill tracking reduces skill-assisted success from 78.1% to 61.7%. Disabling library lifecycle management reduces skill-free success from 72.7% to 53.1%. These are separate ablations measuring different consequences of the framework.

For operators of reusable instruction libraries, the practical implication is a more accountable process for determining which instructions deserve continued retrieval. The limitation is equally operational: TrackRL’s performance-based rules can leave harmful instructions active while evidence accumulates. Statistical utility tracking cannot replace independent safety checks.

A successful execution cannot assign credit to every instruction

Consider an agent that retrieves three instructions before completing a workflow. One instruction may improve its decision, another may repeat knowledge already available in the model, and a third may actually interfere with execution. A log that marks all three as successful provides no way to distinguish these cases.

Task difficulty adds another problem. An instruction retrieved for easy tasks may appear effective simply because those tasks are easier. Comparing its historical success rate with another instruction’s rate would mix the effect of the instruction with differences in the tasks where it appeared.

TrackRL addresses both problems during reinforcement learning.

It maintains two textual skill libraries: task-level planning instructions retrieved at the start of an episode and step-level execution instructions retrieved as observations change. Candidate skills are initially selected through semantic retrieval, then reranked using relevance, estimated utility, and an exploration bonus.

During training, the system generates eight episodes for each sampled task under a common behavior policy. Rather than injecting every selected skill into every episode, it independently randomizes whether each skill is supplied, using an injection probability of 0.5. Separate skill-free episodes are also reserved.

For each selected skill, the framework compares task success among episodes receiving that instruction with success among episodes where it is withheld. Because the comparison occurs within the same task, task difficulty is held constant. Independently randomizing other selected skills also reduces systematic credit assignment to instructions that merely appear alongside a beneficial one.

The resulting skill utility is a weighted treatment-minus-control success-rate difference. Comparisons with larger, better-balanced treatment and control groups receive more weight. Older evidence is discounted as training proceeds, allowing the estimate to respond when the underlying policy changes.

This is a meaningful causal measurement, but its scope is specific. The estimate concerns the effect of injecting a selected skill under the contemporaneous policy, retrieval context, and distribution of other skill assignments. It is not an intrinsic measure of an instruction’s value across every task, model version, or possible skill combination.

A skill can become unnecessary without becoming harmful

Attribution alone does not settle what to remove from a library.

Suppose an instruction initially improves task success. After further training, supplying it no longer makes a measurable difference. That result admits at least two explanations: the model may have become capable of completing the relevant task unaided, or the instruction may never have been reliably useful.

TrackRL therefore maintains two distinct measurements.

Skill utility asks how much success changes when an instruction is supplied. Model mastery asks how successfully the policy completes the corresponding task when that task-level instruction is absent. Mastery is estimated from discounted control-arm outcomes, using a Beta prior to smooth the estimate and quantify uncertainty.

The distinction determines lifecycle actions.

New instructions generated from failed trajectories undergo similarity-based admission screening against existing and previously harmful skills. Once admitted, they receive protected incubation until at least ten eligible within-group comparisons have accumulated and utility uncertainty can be estimated.

A skill is removed as harmful only when its estimated utility plus twice its standard error is below zero, after the minimum evidence requirement is satisfied. This requires evidence of negative contribution, rather than merely observing an unsuccessful execution.

Retirement follows a different rule. A task-level skill can retire when its estimated utility is non-positive and the lower confidence bound on mastery exceeds both an absolute threshold of 0.5 and the relevant task-family baseline. A retired skill may later be regenerated after new failures; harmful skills are recorded to prevent sufficiently similar instructions from being readmitted.

Step-level skills receive utility estimates but do not use the same task-level mastery-based retirement rule.

Library capacity also respects uncertainty. Each pool has a soft capacity of 300 skills, but TrackRL permits temporary overflow instead of evicting instructions that lack sufficient evidence. Eligible non-beneficial skills can be removed to relieve capacity pressure.

These rules treat instruction management as a continuing measurement problem. The appropriate decision depends on the skill’s measured contribution, the policy’s unaided competence, and the strength of the observations supporting both.

Tracking and lifecycle management affect different outcomes

The paper evaluates TrackRL using Qwen3-4B-Instruct-2507 on two interactive benchmarks: ALFWorld, involving textual household tasks, and AppWorld, involving application APIs.

The ALFWorld ablations are particularly informative because they isolate the two central mechanisms.

Removing individual skill tracking, so that tracking operates at the retrieved-bundle level, reduces seen skill-assisted success from 78.1% to 61.7%, a decline of 16.4 percentage points. This supports the value of assigning credit to individual skills rather than treating the retrieved bundle as a single contributor.

A separate ablation disables lifecycle exits while continuing to admit skills. Seen skill-free success falls from 72.7% to 53.1%, a decline of 19.6 points. Within this configuration, managing library exits contributes to the performance of the trained policy even when external skills are absent at evaluation.

The broader benchmark comparisons reinforce these findings, with qualifications.

Under base-model initialization and GPT-5.6-Luna reflection, TrackRL achieves 78.1% seen and 71.9% unseen skill-assisted ALFWorld success, versus 57.0% and 54.7% for D2Skill under the corresponding reflection configuration. Its skill-free results reach 72.7% and 64.1%, compared with 58.6% and 53.0% for plain GRPO.

The advantage is sensitive to configuration. With Gemini-2.5-Flash reflection, the cold-start skill-assisted improvement over D2Skill narrows to 2.4 points on seen tasks and 0.8 points on unseen tasks.

AppWorld provides a more demanding boundary. With base initialization and Luna reflection, TrackRL reaches 23.2% normal-split task goal completion with skills and 22.6% without them, compared with 15.5% skill-free for GRPO. Under SFT initialization, its corresponding results rise to 33.3% and 34.5%. Yet challenge-split scenario goal completion remains only 8.6% with skills and 7.9% without them.

These are benchmark comparisons, not randomized causal estimates of the superiority of the entire training system. Nor do the reported results provide comprehensive independent-run uncertainty estimates for every performance difference.

The library changes as the policy learns

The longitudinal analyses show why library decisions cannot be based on fixed historical success rates.

One harmful step-level skill is removed at training step 91 with estimated utility of −0.11. Another skill, at the task level, is retired at step 39 after its mastery estimate rises from 0.50 to 0.97. A third, step-level skill remains useful, retaining estimated utility of 0.07 after 310 comparisons.

These examples illustrate different lifecycle outcomes rather than proving a universal pattern. Across tracked skills, the share with mastery above 0.5 rises from 35% at step 20 to 41% at step 160, while many utility estimates move toward zero.

Task families also differ. Pick and Cool exhibit retirement patterns consistent with improving unaided competence; Clean continues to benefit from external skills. Pick2 shows policy improvement despite near-zero measured external skill utility.

Under SFT initialization, the library finishes with 212 active skills, comprising 80 task-level and 132 step-level entries. This smaller library reflects a different starting policy and evolving skill requirements, not simply a more aggressive deletion policy.

Crucially, successful skill-free behavior is evidence of task competence, not direct proof that the model has internalized the knowledge represented by a particular instruction. A model can achieve the same outcome through other learned procedures.

Enterprise skill governance needs evidence and independent safeguards

For an operator maintaining reusable instructions in a coding, customer-service, or workflow agent, TrackRL suggests a concrete governance record: which instruction was retrieved, under what policy version and task context, how often it was tested with and without injection, its estimated contribution and uncertainty, and the evidence supporting its lifecycle status.

This could make instruction reviews more informative after model updates. An instruction that previously helped should not retain priority indefinitely without renewed evidence, while an instruction whose contribution disappears should not automatically be classified as harmful.

The paper does not establish that the same procedures will improve deployed enterprise agents. Its experiments cover two simulated environments and one trained policy architecture. The thresholds, retrieval settings, and capacity limits have not been established as generally optimal. Reusing reinforcement-learning rollouts for attribution also does not demonstrate lower end-to-end production costs.

There is a more immediate safety constraint. Generated skills can contain incorrect guidance, unsafe generalizations, or sensitive information from originating trajectories. A potentially harmful instruction may remain active during protected incubation, before sufficient comparative evidence exists for removal.

For safety-sensitive applications, admission controls, content restrictions, sensitive-data protections, and execution permissions therefore need to operate independently of statistical skill performance. Utility-based removal is a quality-management mechanism, not a complete safety system.

TrackRL’s central contribution is to make the continued value of an external instruction measurable as the agent changes. The research establishes a promising way to separate individual contribution from unaided competence and to record why instructions are retained or retired. Whether that discipline improves production reliability and economics requires separate evidence.

Cognaptus: Automate the Present, Incubate the Future.