Better Fit, Worse Next Move
TL;DR for operators When each experiment or simulation is expensive, selecting a surrogate because it fits existing observations well can be the wrong criterion. In the paper’s controlled surrogate ablation, the most flexible model achieves the lowest training error but materially worse prediction away from observed points and much more optimistic estimates of candidate value. ...