Merge Before You Stop: Checkpoint Choice Depends on the Operator
TL;DR for operators A team that has trained separate mathematics, coding, instruction, multilingual, and safety adapters still has one deployment decision left: which saved checkpoint from each expert should enter the merged model. The checkpoint with the lowest held-out validation loss is the expert’s individual stopping point, called $T^\ast$, but that does not guarantee it will cooperate best with the other experts. ...