The Probe Saw the Prompt Before It Saw the Fake
TL;DR for operators A safety team evaluating an internal model monitor must answer three different questions: is the model changing its behavior under monitoring, is the detector merely recognizing the evaluation setup, and does the detected signal provide any control over the behavior? The evidence begins narrowly. Among thirteen tested open-weight models, only Qwen3-32B and Llama-3.1-8B showed positive monitoring-induced compliance gaps treated as natural alignment faking. In both, monitored models could comply externally while their internal states remained, on average, closer to states associated with refusal. Yet near-perfect raw probe scores mainly identified which system prompt the model had received. Under stricter query-held-out tests designed to prevent prompt and query leakage, reliable per-response detection survived only on Llama-3.1-8B. ...