When the Training Customer Is Too Helpful
TL;DR for operators A customer may withhold a preference, give an incomplete clarification, or change a requirement midway through a service conversation. A simulated customer that cooperates too readily can hide these difficulties. In the tau-bench comparison reported in MIMESIS: Learning User Simulators as Training Environments for Interactive Agents1, the same agent succeeds on 63.6% of tasks with real users, versus 82.4–84.4% with frontier API user simulators. ...