The Model Said Its Answer Wouldn't Change
TL;DR for operators An AI assistant’s prediction about its own behavior should not replace an actual behavioral test. In Evaluating and Improving LLM Self-Modeling, Siqi Zeng, Andre N. Assis, and Rowan Wang1 evaluate 18 models on their ability to predict observable properties of their own responses. The highest aggregate skill score is only +0.147 on a scale from -1 to +1, after accounting for trivial prediction strategies. ...