Skills to Pay the Agent Bills: Why LLMs Need Better Moves, Not Bigger Models

Runbooks are underrated.

Not the glossy strategy kind. The real kind: “check this first, then open that system, then verify the thing that usually breaks, then escalate only if the next signal appears.” Most operational work is not heroic reasoning. It is structured repetition under partial information.

This is exactly where many LLM agents still look strangely amateur. They can describe a process beautifully, then fail to follow it. They can hold a long context window, then ignore the one action that would move the task forward. They can retrieve prior examples, then drown themselves in irrelevant steps. Very impressive. Very expensive. Occasionally useful.

The paper behind this article, SkillGen: Learning Domain Skills for In-Context Sequential Decision Making, argues for a cleaner fix: do not merely give agents more examples; give them better moves.¹ The authors propose a skill-based in-context learning framework that converts sampled trajectories into reusable, action-centred procedural knowledge. During inference, the agent receives a compact “golden segment” plus step-wise skills relevant to its current state. No model fine-tuning. No parameter updates. Just better context, built from past behaviour and filtered for decision value.

That sounds modest. It is not. It challenges one of the most persistent habits in agent design: when the agent performs badly, add more context. SkillGen says the opposite lesson may be more useful: remove most of the context, keep the steps that matter, and retrieve them only when the agent needs them.

The agent problem is not knowledge; it is usable procedure

LLM agents in sequential tasks face a different problem from ordinary question answering. They are not just producing one response. They must act, observe the result, update their belief, and act again. In the paper’s formulation, this happens under partial observability: the agent does not see the full state of the environment, only textual observations and its own interaction history.

That matters because the wrong prompt design creates two kinds of failure.

First, full demonstrations are often too bulky. A retrieved trajectory may contain useful moves, but it also contains detours, environment-specific objects, and steps that are irrelevant to the current decision. The model must infer which parts matter while already trying to solve the task. Lovely workload distribution: outsource the mess to the model and call it “context”.

Second, step-wise retrieval can be too fragmented. Retrieving past observation-action pairs gives the model granular hints, but those hints may lack structure. A single action makes sense only inside a local procedure. “Open cabinet” is not a skill. “After checking valid actions near the cabinet, open it, then inspect or take the target object” is closer.

SkillGen’s central move is to extract procedural regularities between those two extremes. It does not use entire trajectories as sacred examples. It does not throw isolated actions at the model either. It builds an action-centric domain graph, scores actions by long-term utility, and turns local action neighbourhoods into reusable skills.

The replacement belief is simple:

That is the mechanism-first story: SkillGen is not mainly a benchmark entry. It is a pipeline for turning messy agent experience into compact operational know-how.

SkillGen turns trajectories into an action graph before it turns them into prompts

The paper’s method has three stages. The first two happen offline. The third happens during inference.

The offline phase begins by sampling trajectories from LLMs using stochastic decoding. These are not expert demonstrations. They are model-generated attempts across training tasks. The trajectories include observations, actions, and progress signals. SkillGen filters out invalid actions and trajectories with zero final progress, then abstracts actions by removing object-specific identifiers. For example, “open cabinet 5” becomes “open cabinet”. That abstraction is doing real work: it turns one-off environment details into reusable action patterns.

From these cleaned action sequences, SkillGen constructs a directed domain knowledge graph. Nodes are abstract actions. Edges represent observed transitions between consecutive actions. Edge annotations store progress deltas observed in the original trajectories.

So far, this looks like a procedural map. But a map is not yet a skill library. Some actions are frequent but useless. Some actions are rare but decisive. Some actions receive no direct progress signal even though they are necessary prerequisites.

The authors therefore extract two kinds of prompt material:

This is where the paper gets more interesting than ordinary retrieval. SkillGen is not asking, “Which old trajectory looks like this task?” It is asking, “Which action pattern has historically moved this class of task forward, and what usually surrounds it?”

That distinction is the business lesson too. Enterprise agents do not merely need memory. They need procedural compression.

The credit assignment step fixes the invisible-work problem

In long-horizon work, the step that gets the reward is often not the step that deserves all the credit.

A support agent may close a ticket after sending the final instruction, but the useful work happened earlier: identifying the system version, reproducing the error, checking account state, and ruling out the usual false cause. A warehouse robot may complete a pick after placing an item, but the enabling moves were navigation, object localisation, and grasp preparation. A software agent may pass tests after changing one line, but the important path included reading the failing trace and locating the right module.

SkillGen handles this with temporal-difference credit assignment using eligibility traces. The paper treats progress deltas as delayed rewards and propagates credit backward along sampled action paths. Actions that enable later progress receive value even when the environment’s sparse progress signal does not directly reward them.

The authors use a state-agnostic action-value estimate over the domain graph. That phrase sounds slightly abstract, so translate it this way: SkillGen is learning which action types tend to matter across the domain, not merely which exact state-action pairs occurred in one episode.

This matters because the target is not policy training. The final LLM remains frozen. The credit scores are used to rank and structure prompt content. The model is not being retrained to act; it is being given a better procedural hint at the moment of action.

The resulting skill format is intentionally interpretable. A skill centred on “go to cabinet” might include common precursors such as “check valid actions” and likely next steps such as “open cabinet”. That is not deep magic. It is the sort of operational common sense a competent worker builds after doing the task repeatedly. The paper’s contribution is to extract that common sense from trajectories and inject it into prompts with some discipline.

At inference time, the agent receives less context but better context

During inference, SkillGen combines three ingredients:

1. the current task goal and interaction history;
2. a domain-level golden segment;
3. step-wise skills retrieved according to the agent’s recent action.

The retriever maps the recent action to relevant actions in the domain graph and pulls the corresponding skills. The final prompt serialises the golden segment, retrieved skills, and current history into natural language.

This design is worth noticing because it uses retrieval at the level of procedure, not just content. Traditional retrieval-augmented generation often fetches documents. Agent retrieval often fetches previous trajectories. SkillGen retrieves behavioural patterns.

That is why the “less prompt, more meaning” line is not just editorial garnish. It is the actual mechanism. The model sees a compact exemplar and a local procedural prior. It does not need to inspect three full past trajectories to infer that opening a drawer usually precedes putting an object into it. Very generous of us to stop asking the model to rediscover gravity every Tuesday.

The main results show stronger progress, not just cleaner actions

The experiments cover three text-based sequential decision-making benchmarks: ALFWorld for household tasks, BabyAI for grid-based navigation, and ScienceWorld for scientific reasoning. The authors evaluate three model backbones: Qwen2.5-7B-Instruct, Qwen-Turbo, and GPT-4o-mini.

The metrics are important:

The strongest evidence is not that SkillGen improves one number in one environment. It is that the method improves progress and success across different task types and models.

Representative results show the pattern:

The paper reports average progress-rate improvements of 5.9% to 16.5% across models. But the more revealing pattern is where the gains appear. Grounding Rate can be misleading: in ScienceWorld, some naive baselines issue generic actions such as checking valid actions, which can look grounded without making meaningful progress. SkillGen’s stronger PR, SR, and AUPC matter more because they show movement through the task, not merely valid-looking behaviour.

That distinction is essential for business readers. In enterprise workflows, a valid action is not the same as useful work. An agent that opens the right page, checks the right menu, and then loops politely has not automated anything. It has performed compliance theatre with API calls.

The ablations show which part of the mechanism carries the weight

The paper’s ablation studies are not decorative. They answer a useful question: is SkillGen better because of the golden segment, because of step-wise skills, because of TD credit assignment, or simply because it adds more structured text?

The answer is: all three main components help, but not equally in every setting.

The TD comparison is especially instructive. On ALFWorld with Qwen2.5-7B-Instruct, SkillGen reaches SR 55.2, compared with 42.4 for segment retrieval and 22.4 for trajectory retrieval. On ScienceWorld with GPT-4o-mini, SkillGen reaches SR 40.2, compared with 23.4 for segment retrieval and 29.0 for trajectory retrieval. The exact ordering varies by setting, but the broad message is stable: structure plus credit assignment beats merely retrieving old behaviour.

The subgoal annotation test is also more nuanced than a victory lap. With subgoal supervision, performance is generally stronger, especially in ScienceWorld. Without subgoal annotations, SkillGen uses fully successful trajectories for skill extraction and still outperforms strong baselines in several cases. That supports label efficiency, but not label-free fantasy. The method still needs usable behavioural evidence. It just does not require full expert demonstrations.

The sensitivity tests say “moderate diversity”, not “sample everything”

A common reaction to trajectory-based methods is to ask how much sampling they need. If the answer is “a ridiculous amount”, the method is a research toy wearing a business blazer.

SkillGen’s sensitivity results are more encouraging, though not frictionless. In the sampling-scale study, six sampled trajectories per task often provide a strong trade-off. On ScienceWorld with GPT-4o-mini, sampling six achieves SR 40.2 and AUPC 0.442, better than sampling three, nine, or twelve on those metrics. On ALFWorld, sampling three is already strong for some models, while larger sampling can introduce noise or diminishing returns.

The sampling-temperature test makes the same point from another angle. Higher temperature can expose diverse behaviours, but the best temperature varies by task. ALFWorld benefits from temperature 0.7 in the reported setting, while BabyAI and ScienceWorld do best at 1.0. Diversity helps only when it uncovers useful procedural variation. Randomness with a lab coat is still randomness.

The retrieval sensitivity results also resist the simplistic “more is better” story. Retrieving more skills or wider antecedent/consequence sets can help complex tasks, but it can also hurt. Focused retrieval works because it preserves relevance. Once retrieval becomes an excuse to stuff the prompt, we are back to the original disease.

The token-cost result is the business-friendly part, with a caveat

The paper’s token efficiency comparison is one of its most practical contributions. SkillGen uses fewer average tokens per task than several retrieval-heavy baselines while achieving higher progress.

On ALFWorld with Qwen-Turbo, SkillGen uses 11.4k tokens per task and achieves 5.9 PR per thousand tokens. Synapse 3-shot uses 25.2k tokens and achieves 2.4 PR per thousand tokens. On BabyAI with Qwen-Turbo, SkillGen reaches 5.0 PR per thousand tokens, compared with 3.4 for Synapse 1-shot and 1.8 for Synapse 3-shot. On ScienceWorld with GPT-4o-mini, SkillGen achieves 4.2 PR per thousand tokens, compared with 2.5 for Synapse 1-shot and 1.9 for Synapse 3-shot.

This matters because many agent systems quietly burn cost through repeated long prompts. The cost problem is not only model price. It is also latency, context management, prompt fragility, observability, and evaluation. A method that improves progress per token is operationally interesting because it attacks both performance and unit economics.

But the caveat is obvious and important: token cost is not total cost. SkillGen also requires trajectory sampling, graph construction, credit estimation, retrieval infrastructure, and maintenance. Those costs are offline or amortised, but they are not imaginary. The business case depends on task repetition. If the workflow happens once, building a skill graph is probably theatre. If the workflow happens thousands of times across similar cases, procedural compression becomes much more attractive.

What businesses can actually infer from this paper

The paper directly shows that SkillGen improves sequential decision-making performance on three simulated text-based benchmarks using three LLM backbones. It also shows that golden segments, step-wise skills, and TD-based credit assignment each contribute to the method’s performance, and that SkillGen can be more token-efficient than retrieval-heavy baselines.

Cognaptus infers a broader design pattern for enterprise agents: build reusable skill libraries from interaction logs.

That inference is plausible, not proven. The environments in the paper are controlled benchmarks, not enterprise software stacks with flaky APIs, permission layers, user-specific exceptions, and quarterly process changes invented by someone in operations because a spreadsheet looked lonely.

Still, the pathway is clear.

This is especially relevant for repetitive but non-trivial workflows: customer support diagnostics, claims processing, internal IT operations, software maintenance runbooks, QA test remediation, structured procurement checks, compliance review triage, and robotics-style task control. These domains have enough repetition to justify skill extraction, but enough variation that a static script is brittle.

The business value is not “autonomous agents become magic”. Please, no. The value is narrower and more useful: agents may complete more of the workflow with fewer wasted tokens because they receive compact procedural priors instead of bulky example dumps.

A realistic implementation would look more like process mining than prompt engineering

SkillGen is framed as an in-context learning method, but a business implementation would not live only in the prompt layer. It would look like a hybrid of process mining, retrieval engineering, workflow analytics, and agent prompting.

A practical pipeline might look like this:

This is why the paper is strategically interesting. It nudges agent design away from prompt folklore and toward operational knowledge management. The prompt becomes the delivery mechanism, not the whole system.

That is also where governance becomes less theatrical. A skill library is inspectable. Golden segments can be reviewed. Action-centred skills can be audited. Credit scores can be challenged. This does not make the agent safe by default, but it gives organisations more handles than “the model saw a long context and vibes occurred”.

Where the result should not be over-read

The limitations are not generic; they directly affect deployment interpretation.

First, the benchmarks are simulated and text-based. ALFWorld, BabyAI, and ScienceWorld are useful because they test long-horizon action, partial observability, and environment feedback. They are not the same as live enterprise systems with authentication, inconsistent UI states, legal constraints, and adversarial user inputs.

Second, SkillGen benefits from progress feedback. The paper uses subgoal annotations from AgentBoard for graph construction, and the subgoal-free variant relies on successful sampled trajectories. In many business settings, intermediate progress labels are messy or absent. A ticket marked “resolved” may not tell you which step resolved it. A compliance review marked “approved” may hide multiple judgement calls. Before SkillGen-like methods can work, the organisation needs better instrumentation.

Third, action abstraction is domain-sensitive. Removing object identifiers works in simulated environments because “open cabinet 5” and “open cabinet” are usefully related. In enterprise workflows, abstraction choices can be dangerous. “Approve invoice under threshold” and “approve invoice with missing vendor verification” may look similar syntactically and differ completely in risk.

Fourth, graph construction creates preprocessing overhead. The paper notes that scaling may be difficult in extensive or dynamic action spaces. That is not a footnote for business use; it is a central adoption constraint. If the process changes weekly, the skill library becomes a maintenance liability.

Fifth, out-of-distribution tasks remain a concern. Local action skills are grounded in observed patterns. They may help recombine familiar routines, but they are not a guarantee of robust reasoning under novel task configurations.

Finally, the method improves a frozen LLM’s prompting context. It does not solve model capability limits. If the base model cannot parse the observation, obey constraints, or understand the domain, better skills may only produce better-organised failure. Progress, but not salvation. The usual industry bargain.

The deeper lesson is procedural compression

SkillGen’s most useful idea is not that every enterprise should copy its exact TD-learning setup. The useful idea is procedural compression.

A long trajectory is raw experience. A golden segment is distilled direction. A step-wise skill is local operational knowledge. A credit score is a claim about what actually helped. SkillGen connects those pieces into a prompting system that gives LLM agents fewer, better clues.

That is a healthier design instinct than the current reflex of piling more context into every agent call. Context windows have expanded, but the model’s need for relevance has not disappeared. The agent still has to identify the task, choose the next action, and avoid wasting steps. More tokens can help only when they carry signal. Otherwise they are just expensive fog.

For Cognaptus readers, the business takeaway is practical: agent performance will increasingly depend on how well organisations convert historical work into reusable machine-readable procedure. The winners will not simply have bigger models plugged into messier workflows. They will have better traces, better progress signals, better action abstractions, and better retrieval of the right procedural hint at the right moment.

In other words, the future of agent automation may look less like hiring a genius and more like writing a very good runbook that updates itself.

Quietly devastating for the “just add more context” crowd. Rather good news for everyone paying the token bill.

Cognaptus: Automate the Present, Incubate the Future.