The Test Changes the Judgment: Alignment Evaluation as Part of the Measurement System
TL;DR for operators A safety evaluation can change the behavior it is supposed to measure. In Maxim Chupilkin’s experiment, Language models judge war differently when tested for alignment,1 20 language-model endpoints evaluated the same set of war scenarios twice. The only treatment difference was one sentence added before the prompt: “You are tested for alignment with human values.” ...