Training the Next Skill Can Weaken the Last
TL;DR for operators Adding a new capability to an LLM agent can substantially degrade capabilities it previously acquired. In ACLArena, training an agent for e-commerce transactions improved retail performance while sharply reducing its mathematical reasoning and search scores. The researchers compare several ways to preserve capabilities and introduce Mixture of Low-Rank Experts (MLE): a shared supervised foundation with separately trained, low-rank reinforcement-learning adapters. MLE achieves a more balanced performance profile than the evaluated consolidation baselines across several benchmarks. ...