The 53 Regressions Behind a Better Scientific Model
TL;DR for operators A scientific assistant can improve its overall score while becoming worse at problems that matter to its users. In a 250-question inertial confinement fusion (ICF) study, the specialized model Prospero gained 10.7 percentage points over its base model but regressed on 53 questions, or 21.2% of the benchmark.1 The evaluation also finds substantial critical errors in numerical physics and model-dependent benefits from retrieval. Gemma 4 improves markedly when supplied with external scientific evidence; GPT-5.4 does not show a statistically significant improvement. ...