A Smaller Agent Is Not the Same Agent: Training for Deployment Perturbations
TL;DR for operators Wang et al. study a deployment problem that is easy to underestimate: an RL-trained agent can work well before compression or modification and then fail badly after its policy is perturbed.1 Their proposed method, Stable Perturbation-Robust Policy Optimization (SPrPO), trains the agent under perturbations while allocating less noise to locally sensitive FFN channels and adjusting total perturbation strength using a gradient-based improvement proxy. ...