The Grader Is Part of the System: What RLVR Verifier Audits Reveal
TL;DR for operators A mathematical verifier is not an invariant scoring utility. Esther Xin’s Where the Verifier Fails1 audits four implementations and configurations using 307,420 verifier verdicts and finds acceptance of mathematically equivalent answers ranging from 53.8% to 95.2%. Two extraction configurations of the same math-verify library disagree on 49.9% of equivalent cases where both return a verdict. ...