Training the Next Skill Can Weaken the Last

TL;DR for operators

Adding a new capability to an LLM agent can substantially degrade capabilities it previously acquired. In ACLArena, training an agent for e-commerce transactions improved retail performance while sharply reducing its mathematical reasoning and search scores.

The researchers compare several ways to preserve capabilities and introduce Mixture of Low-Rank Experts (MLE): a shared supervised foundation with separately trained, low-rank reinforcement-learning adapters. MLE achieves a more balanced performance profile than the evaluated consolidation baselines across several benchmarks.

For organizations extending agents into new workflows, the findings support three practices: test existing capabilities after each training stage, consider isolated adapters where workflows are distinguishable, and validate expert selection before deployment. The evidence comes from one foundation model and one principal training sequence. Routing under ambiguous conditions, longer training histories, and full deployment costs remain unresolved.

A Retail Upgrade Can Regress Reasoning and Search

Consider an enterprise assistant already capable of answering research questions and solving mathematical problems. The organization extends it to customer-service transactions, training the same model to interact with retail APIs, retrieve account information, and complete customer requests.

The new training succeeds. The assistant becomes substantially better at retail transactions. Yet it may become worse at the reasoning and search tasks it previously handled.

This is the problem investigated in ACLArena, a study of sequential agent post-training across heterogeneous capabilities.¹ The researchers train Qwen3-8B-Base through four stages: mathematical reasoning, search, e-commerce, and instruction following. They evaluate the model after successive stages rather than assuming that later checkpoints retain everything learned earlier.

The most revealing transition occurs when search training is followed by e-commerce training.

Source: ACLArena, Section 3, Table 1. Values are reported benchmark scores under the study’s evaluation protocols.

Retail task success rises by 32.0 points, but mathematical reasoning loses approximately three-quarters of its preceding score, while Natural Questions falls by 30.6 points.

The subsequent instruction-following stage partially reverses these regressions: AIME26 recovers to 10.21 and Natural Questions to 33.5. Neither returns to its earlier peak, and retail performance declines from 34.0 to 29.6.

The resulting checkpoint has acquired all four kinds of training experience without retaining each capability at its strongest observed level.

For an organization updating a production agent, successful learning of the newest workflow therefore cannot serve as evidence of overall improvement. Previous capabilities must remain part of the acceptance tests.

Most Token Predictions Remain Stable While Capabilities Change

Why does learning a new workflow interfere with established behavior?

The researchers examine parameter updates and token-level predictions across training stages.

Their parameter-space analysis finds that heterogeneous tasks induce partially incompatible optimization directions. Particularly revealing are the isolated reinforcement-learning increments: Appendix G reports absolute pairwise cosine similarities no greater than 0.004, indicating nearly orthogonal update directions under the evaluated configuration.

This does not mean every parameter update directly cancels an earlier one. It indicates that specialized training objectives can move the model toward substantially different parameter configurations. When subsequent stages repeatedly modify shared parameters, preserving earlier capabilities becomes difficult.

The token-level analysis reveals a less intuitive pattern.

The researchers replay fixed token sequences through successive checkpoints and measure whether each model retains the original reference model’s top-ranked prediction. Positions are grouped by predictive entropy, a measure of uncertainty in the reference model’s token distribution.

Source: Appendix H, Table 7. Low entropy means H < 0.5 nats; high entropy means H ≥ 0.5. Consistency measures agreement with the mathematical-training reference checkpoint on fixed token histories.

Across these benchmarks, more than 90% of analyzed token positions fall into the low-entropy group and retain highly consistent predictions. Changes are disproportionately concentrated in the relatively small high-entropy group.

This helps explain why aggregate prediction stability can coexist with large task-level regressions. A model may preserve nearly all predictable continuations while changing its choices at uncertain positions that affect later reasoning or actions.

The analysis is diagnostic rather than causal. It does not demonstrate that entropy causes forgetting or establish that every changed high-entropy token is consequential. It identifies where prediction instability is concentrated.

Consolidating Specialists Does Not Guarantee Their Combined Strengths

If successive training damages earlier capabilities, one alternative is to preserve specialized checkpoints and consolidate their behaviors afterward.

ACLArena compares three consolidation approaches.

Multi-teacher mixed on-policy distillation (MMOPD) trains a shared student on trajectories it generates itself, using domain-specific frozen teachers. Its loss combines reverse KL with an entropy-gated, top-q-truncated forward-KL correction. The additional term helps preserve plausible alternatives at uncertain token positions. This modification addresses training collapse observed with pure reverse-KL distillation in the evaluated search setting.

Self-distilled fine-tuning (SDFT) uses filtered successful trajectories generated by specialist models. Rather than interacting with frozen teachers on newly generated student trajectories, it trains through supervised token likelihood on a consolidated dataset.

Model merging combines parameter updates from previous checkpoints. It requires no additional learning stage for the merge itself, but successful interpolation depends on the compatibility of the constituent parameter configurations.

The main comparison shows how differently these approaches distribute performance.

Source: Sections 4–6, Table 2. Results are main-comparison point estimates; benchmark scales and evaluation protocols differ.

MMOPD restores much of the lost reasoning and search performance while largely preserving instruction following. SDFT produces stronger multi-hop search results but reduces IF-Eval from 84.8 to 53.6 and retail performance from 29.6 to 22.3.

Model merging also redistributes capability rather than preserving every specialist’s strongest performance. Its appendix coefficient sweep is a sensitivity analysis showing how weighting checkpoints changes these trade-offs, not evidence of a universally preferable merge.

The accompanying ablations investigate why consolidation is uneven. Single-teacher distillation demonstrates targeted search recovery, while SDFT initialization and sampling comparisons show that the trajectory mixture affects outcomes. These experiments help explain the mechanisms but do not establish a consolidation strategy that dominates across every task.

The central difficulty remains: different capabilities are being compressed into parameters that must serve competing objectives.

MLE Keeps Task-Specific Reinforcement Learning Separate

The researchers’ proposed architecture, Mixture of Low-Rank Experts, responds by separating transferable behavior from task-specific refinement.

The design draws on another experimental observation: supervised fine-tuning generally produces parameter updates approximately three to seven times larger than those from reinforcement learning and mixed on-policy distillation in the evaluated pipelines.

Supervised learning provides essential behavioral initialization, including valid tool syntax and interaction protocols. Reinforcement learning then makes smaller, task-dependent policy improvements. Experiments without supervised initialization reinforce the importance of establishing agent interaction behavior before optimizing task outcomes.

MLE exploits these complementary functions through two stages:

1. Shared behavioral learning: train a common SDFT backbone using filtered trajectories from task specialists.
2. Isolated policy refinement: freeze the shared backbone and independently train one LoRA adapter per environment using reinforcement learning.

Its effective task-specific parameters can be expressed as:

Here, $\theta_{\mathrm{shared}}$ is the frozen common model and $\Delta\theta_k$ is the low-rank reinforcement-learning adaptation for environment $k$.

Training a subsequent expert no longer modifies the previous experts’ parameters. At inference, observable environment information—including instructions and available tool schemas—determines which adapter is activated.

MLE achieves 49.7 on Natural Questions, 32.9 on Tau3-Retail, and 85.0 on IF-Eval. It also records the highest scores in the main comparison on GPQA (42.4), single-hop search (57.7), multi-hop search (38.6), and Tau3-Mock (66.7).

These results support parameter isolation as a promising architecture for retaining heterogeneous capabilities.

They do not demonstrate universal superiority. SDFT obtains a higher AIME26 score than MLE, and the separately trained mathematical expert reaches 25.83 compared with MLE’s 21.04.

MLE also has an architectural distinction: it does not produce one independently sufficient shared policy containing every specialist’s best behavior. Its performance depends on selecting the appropriate expert.

Expert Selection Becomes a Deployment Requirement

For an organization operating agents in clearly separated environments, the MLE design offers a plausible way to extend capabilities without repeatedly overwriting previous task-specific adaptations.

A research assistant, retail transaction agent, and instruction-following service could share substantial behavioral competence while retaining separately optimized refinements.

However, their environments must be identifiable at execution time.

ACLArena uses observable context to select experts. It does not establish reliable routing when user requests are ambiguous, environments change during an episode, or successful execution requires combining several adapters.

The research also uses one principal foundation model, Qwen3-8B-Base, and one four-stage curriculum. Longer sequences, alternative training orders, and the complete memory, latency, and maintenance costs of expert deployment remain insufficiently tested.

Cognaptus inference: For organizations considering this architecture, evaluation should cover three separate requirements: retention of previously supported tasks, accuracy of expert selection under actual workflow conditions, and the incremental operational cost of maintaining and serving specialist adapters.

These requirements extend beyond the paper’s demonstrated benchmark performance. They are deployment conditions to validate, not measured business outcomes reported by ACLArena.

Capability Retention Must Be Measured Independently

ACLArena shows that acquiring a new agent capability and retaining previous capabilities are separate outcomes. Sequential training can improve the latest task while substantially weakening earlier ones; distillation, replay, and model merging can recover performance without eliminating trade-offs.

A frozen shared backbone with isolated low-rank experts offers a more effective balance across several evaluated tasks because new reinforcement-learning updates no longer overwrite earlier specializations.

For practical adoption, this shifts attention from the performance of the latest checkpoint to the capability profile of the complete deployed system, including its routing behavior. The strongest result is not that continual learning has been solved, but that separating shared learning from specialized adaptation can substantially improve retention under identifiable operating conditions.

Cognaptus: Automate the Present, Incubate the Future.