A Correct Answer Is Not a Complete Model Profile
TL;DR for operators A model’s correctness score tells you whether its answers passed a test, but discards information about how it responds. BULLPEN shows that retaining answer text substantially improves prediction of behavioral, stylistic, and provenance characteristics across 260 open-weight models. Its answer-only representation achieves 0.48 normalized predictive skill on 20 new tasks, compared with 0.35 for the strongest prior encoder. ...