Right Answer, Wrong Evidence: A Deployment Gate for Grid-Diagnosis LLMs
TL;DR for operators A grid operator may see topology, live measurements, and an incident narrative all point to the same diagnosis. The decision is not only whether the answer is correct, but whether the model relied on evidence that the diagnostic task permits it to use. In the study, shortcut incident text produced a mean signed utility effect of +0.062 even though its preregistered engineering importance was zero. The model therefore became more accurate by using evidence that should not have determined the answer. Accuracy and a plausible explanation cannot reveal that divergence on their own. ...
