Same Verdict, Different Evidence: Auditing AI Judges
TL;DR for operators Two AI judges can select the same correct answer while providing explanations of substantially different reliability. Treating those judgments as equally valuable overlooks information that matters for evaluation, post-training, and decision auditing. DeCert, introduced by Sun and colleagues,1 evaluates whether a justification uses valid evidence, addresses material issues, and adequately supports its verdict. It also generates training preferences between explanations that reach the same correct conclusion. ...